not sure what PocketOS does or why your whole dataset would be a single volume without a clear separation between application and automotive data. how are you decoding VINs?
An AI agent deleted our production database. The agent's confession is below
11–20 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#12Someone trusted prod database to an llm and db got deleted. This person should never be trusted with computers ever again for being illiterate
If the account is to be believed that's not what happened. They asked the LLM to do something on the staging environment, it chose to delete a staging volume using an API key that it found. But the API key was generated for something else entirely and should not have been scoped to allow volume deletions - and the volume deletion took out the production database too.
The LLM broke the safety rules it had been given (never trust an LLM with dangerous APIs). *But* they say they never gave it access to the dangerous API. Instead the API key that the LLM found had additional scopes that it should not have done (poster blames Railway's security model for this) and the API itself did more than was expected without warnings (again blaming Railway).
Re: An AI agent deleted our production database. The agent's confession is below
#13Absolutely zero sympathy. You’re responsible for anything an agent you instructed does. Allowing it to run independently is on you (and all the others doing exactly this). This is only going to become more and more common.
Re: An AI agent deleted our production database. The agent's confession is below
#14The details of the story are interesting. Backups stored on the same volume is an interesting glitch to avoid. Finding necessary secrets wherever they happen to be and going ahead with that is the kind of mistake I've seen motivated but misguided juniors make. Strange how generated code seems to have many security failings, but generated security checks find that sort of thing.
Re: An AI agent deleted our production database. The agent's confession is below
#15The way this is written gives me the impression they don’t really understand the tools they’re working with.
Master your craft. Don’t guess, know.
Re: An AI agent deleted our production database. The agent's confession is below
#16There is something darkly comical about using an LLM to write up your “a coding agent deleted our production database” Twitter post. On another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that ma…
Twitter users get paid for these 'articles' based on engagement, correct? That may be the reason why it is so dramatized.
Re: An AI agent deleted our production database. The agent's confession is below
#17There is something darkly comical about using an LLM to write up your “a coding agent deleted our production database” Twitter post. On another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that ma…
> a misunderstanding in the users mind about how the agent work
On top of that the agent is just doing what the LLM says to do, but somehow Opus is not brought up except as a parenthetical in this post. Sure, Cursor markets safety when they can't provide it but the model was the one that issued the tool call. If people like this think that their data will be safe if they just use the right agent with access to the same things they're in for a rude awakening.
From the article, apparently an instruction:
> "NEVER FUCKING GUESS!"
Guessing is literally the entire point, just guess tokens in sequence and something resembling coherent thought comes out.
Re: An AI agent deleted our production database. The agent's confession is below
#18Interesting story. But despite Cursors or Railways failure, the blame is entirely on the author. They decided to run agents. They didnt check how Railway works. They relied on frontier tech to ship faster becsuse YOLO.
I really feel sorry for them, I do. But the whole tone of the post is: Cursor screwed it up, Railway screwed it up, their CEO doesnt respond etc etc.
Its on you guys!
My learning: Live on the cutting edge? Be prepared to fall off!
Re: An AI agent deleted our production database. The agent's confession is below
#19The way this is written gives me the impression they don’t really understand the tools they’re working with. Master your craft. Don’t guess, know.
> Master your craft. Don’t guess, know.
You mean add that to my prompt right ?
Re: An AI agent deleted our production database. The agent's confession is below
#20This is hilarious.