511–520 of 1001 posts
The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…
> Because Railway stores volume-level backups in the same volume Anyone familiar with Railway no why this is done this way? This seems glaringly bad on its face.
There is something darkly comical about using an LLM to write up your “a coding agent deleted our production database” Twitter post. On another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that ma…
If you ask humans to explain why we did something, Sperry's split brain experiment gives reason to think you can't trust our accounts of why we did something either (his experiments showed the brain making up justifications for decisions it never made) Bit it can still be useful, as long as you interpret it as "which stimuli most likely triggered the behaviour?" You can't trust it uncritically, but models do sometime…
Llms are just too creative. They will explore the search space of probable paths to get to their answer. There's no way you can patch all paths
We had to build isolation at the infra level (literally clone the DB) to make it safe enough otherwise there was no way we wouldn't randomly see the DB get deleted at some point
The biggest rule-break was done, not by the agent or infra company, but by the person who gave such elevated authorization (API key) to an autonomous bot.
Earlier quoted context omitted.
None of the developers that I’ve worked with have had the hemispheres of their brains severed. I suspect this is pretty rare in the field.
This still doesnt stop post ad hoc explanations by humans.
The way this is written gives me the impression they don’t really understand the tools they’re working with. Master your craft. Don’t guess, know.
Top user of cursor. Build AI Agents and LLMs. Very aware of limitations and a senior software dev. Cautionary tale for other builders. DYOR.
Anything else is just gambling.
The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…