Earlier quoted context omitted.
> Master your craft. Don’t guess, know. You mean add that to my prompt right ?
"Make no mistakes"
> "NEVER FUCKING GUESS!"
An AI agent deleted our production database. The agent's confession is below
61–70 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#62It seems like the most unreasonable thing happening here is Railway's backup model and lack of scoped tokens. On the agent side of things, how would one prevent this, short of manually approving all terminal commands? I still do this, but most people who use agents would probably consider this arcane. (Let's suppose the agent did need an API token to e.g. read data).
Wrapper around the function call. Don't give it the token itself but a limited set of fixed functions to create domains (their use case according to the post).
Additionally give it a similar restricted way to "delete" domains while actually hiding them from you. If you are very paranoid throw in rate limits and/or further validation. Hard limits.
Yes this requires more code and consideration but well that's what the tools can be fully trusted with.
Re: An AI agent deleted our production database. The agent's confession is below
#63That happens if you aggressively buy into the latest tech without thinking about if you really need it.
Why do you need an AI agent for working on a routine task in your staging environment?
"Never send a machine to do a human's job."
Re: An AI agent deleted our production database. The agent's confession is below
#64Plenty of everyone doing it wrong, but the most WTF of all the WTFs is the backup storage.
Put your backups in S3 *versioned* storage on a different AWS account from your primary, and set some reasonable JSON lifecycle rule:
"NoncurrentVersionExpiration": {
"NoncurrentDays": 30,
"NewerNoncurrentVersions": 3
}
Re: An AI agent deleted our production database. The agent's confession is below
#65"We gave DROP grants in prod to the user running AI agents irresponsibly at our company, and the expected happened." FTFY.
In seriousness, RBAC, sandboxing, any thing but just giving it access to all tools with the highest privileges...
Re: An AI agent deleted our production database. The agent's confession is below
#66That’s our new reality. Some people seem not to not grasp that all those AIs are just mathematical models producing the next most statistically likely token. It doesn’t feel anything, nor does it care about what it does. The difference between test and production environment is just a word. That, in contrast to a human who would typically have a voice in the back of his head “this is production DB, I need to be careful”.
Re: An AI agent deleted our production database. The agent's confession is below
#67not sure what PocketOS does or why your whole dataset would be a single volume without a clear separation between application and automotive data. how are you decoding VINs?
Makes me wonder also about multi-tenancy. If all customer information is in single volume. How big risk they put on their customers on their most business critical and proprietary data to leak other competitiors?
Re: An AI agent deleted our production database. The agent's confession is below
#68I don’t see the problem here. These people will be pushed out of the industry quickly and their business taken by other people, who are using agents, but are smart enough to run them sandboxed without any permission to production or even dev data/systems.
Re: An AI agent deleted our production database. The agent's confession is below
#69The way this is written gives me the impression they don’t really understand the tools they’re working with. Master your craft. Don’t guess, know.
> Master your craft. Don’t guess, know. You mean add that to my prompt right ?
If you also add "don't break the previous rule", you should be 100% safe.
Re: An AI agent deleted our production database. The agent's confession is below
#70Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…
In AWS eg. bucket can be deleted only when empty. Deleting all files first is your confirmation.