Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

Post reply on HN
Print view View original

741–750 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#741

Post by palmotea »

cwsx wrote: post #695
6r17 wrote: post #681

Earlier quoted context omitted.

On a less dramatic pissed (rightfully) reading ; I have found that if you do give the capability to a LLM to do something ; it will be inclined to see this as an option to solving what it what asked to ; but then giving the instruction by negative present very poor results whereas the same can be driven by a positive one ; a "don't delete the database" becomes "if you want to reset the database you have a tool that y…

I tell people to treat LLM's like a toddler (albeit a very capable toddler). Do kids learn well when you only tell them what NOT to do? Of course not! You should be explaining how to do things correctly, and most importantly the WHY, as well as providing examples of both the "correct" and "incorrect" ways (also explaining why an example is incorrect).

> I tell people to treat LLM's like a toddler (albeit a very capable toddler).

Bbbbut a guy from Anthropic, just this last Friday, told me to think of Claude as my "brilliant coworker"! Are you telling me that's not true!?

Re: An AI agent deleted our production database. The agent's confession is below

#742

Post by lmm »

827a wrote: post #261

The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…

> Anyone who would follow a mistake like that up with demanding a confession out of the agent is not mature enough to be using these tools.

Anyone like that is not mature enough to be managing humans. I'm glad that these AI tools exist as a harmless alternative that reduces the risk they'll ever do so.

Re: An AI agent deleted our production database. The agent's confession is below

#743

Post by dankwizard »

This post is so funny.

Firstly, blaming AI at the same time using AI to construct your whole post - Priceless. Loving it.

Secondly - This entire article reeks of "It's not our fault, you guys have failed us at every step" when in reality you let AI run reckless.

I don't want to say deserved it but like, you knew the risks,

Re: An AI agent deleted our production database. The agent's confession is below

#744

Post by palmotea »

coldtea wrote: post #449
827a wrote: post #261

The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…

> Anyone who would follow a mistake like that up with demanding a confession out of the agent is not mature enough to be using these tools. Lord, even calling it a "confession" is so cringe. The agent is not alive. The agent cannot learn from its mistakes The problem is millions of years of evolutionary wiring makes us see it as alive. Even those mature enough to understand the above on the conscious level, would sti…

> The problem is millions of years of evolutionary wiring makes us see it as alive. Even those mature enough to understand the above on the conscious level, would still have a subconscious feeling as if it's alive during interactions, or will slip using agency/personhood language to describe it now and then.

Also four (4) whole years of propaganda, which includes UX patterns and RLHF optimizations to encourage us to interact with it like a person.

Re: An AI agent deleted our production database. The agent's confession is below

#745

Post by lmm »

amluto wrote: post #214
vidarh wrote: post #93

Earlier quoted context omitted.

If you ask humans to explain why we did something, Sperry's split brain experiment gives reason to think you can't trust our accounts of why we did something either (his experiments showed the brain making up justifications for decisions it never made) Bit it can still be useful, as long as you interpret it as "which stimuli most likely triggered the behaviour?" You can't trust it uncritically, but models do sometime…

Humans can do one thing that AI agents are 100% completely incapable of doing: being accountable for their actions.

What does that actually mean in practice? You can yell at human if it makes you feel better, sure, but you can do that with an AI agent too, and it's approximately as productive.

Re: An AI agent deleted our production database. The agent's confession is below

#747

Post by kuratkull »

Terr_ wrote: post #472
gigatree wrote: post #333

Earlier quoted context omitted.

He’s not necessarily anthropomorphizing it, he’s showing that it went against every instruction he gave it. Sure concepts like “confession” technically require a conscious mind, but I think at this point we all know what someone means when they use them to describe LLM behavior (see also “think”, “say”, “lie” etc)

> He’s not necessarily anthropomorphizing it, he’s showing that it went against every instruction he gave it. It's deeper than that, there are two pitfalls here which are not simply poetic license. 1. When you submit the text "Why did you do that?", what you want is for it to reveal hidden internal data that was causal in the past event. It can't do that, what you'll get instead is plausible text that "fits" at the e…

The best answer so far. It describes exactly what was going on. LLM users should read it twice, especially if "confession" didn't make your brain hurt a bit.

Re: An AI agent deleted our production database. The agent's confession is below

#748

Post by lmm »

fluoridation wrote: post #469
vidarh wrote: post #417

Earlier quoted context omitted.

Sperry's experiments makes it quite clear that the comparison is not nonsensical: humans can't reliably tell why we do things either. It is not imbuing AI with anything more to recognise that. Rather pointing out that when we seek to imply the gap is so huge we often overestimate our own abilities.

Humans at least have a mental state that only they are privy to to work from, and not just their words and actions. The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to.

> Humans at least have a mental state that only they are privy to to work from

Maybe. How do you tell? What would you expect to be different if they didn't?

> The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to.

Insight is not solely a function of available input information. Arguably being able to search and extract the relevant parts is a far more important part of having insights.

Re: An AI agent deleted our production database. The agent's confession is below

#749

Post by Chris660 »

yegle wrote: post #106

AFAKIT the built-in backup of a managed database will be gone if the database is deleted. This is true in AWS and GCP. I still don't know why the product manager would decide this is a good UX.

IIRC in AWS you have the option to create a "final" snapshot of the DB instance when deleting it. I'm pretty sure that's the default behaviour when using the web console, but may merely be an option in the API interface.
Post reply on HN