151–160 of 167 posts
I’m not going to “defend” the LLM here but this: > I forgot to use the state file, as it was on my old computer indicates that this person did not really know what they were doing in the first place. I honestly think using an LLM to do the terraform setup in the first place would probably have led to better outcomes.
No staging environment? No prior attempt to follow best practices (e.g. deletion protection in production)? Nor manual gating of production changes? No attempt to review Claude's actions before performing them? No management of Terraform state file? No offline backups? And to top it off, Claude (the supposed expert tool) didn't repeatedly output "Are you insane? No, I'm not working on that." - Clearly Claude wasn't p…
Considering engineers have made similar mistakes I’m not so sure that’s a great razor, haha
Putting yourself in a situation where this could happen is kinda insane, right? Could be something I'm missing. I can't think of any specific example where I would let any agent touch a production environment, the least of which, data. AI aside, doing any major changes makes sense to do in a dev/staging/preview environment first. Not really sure what the lesson would be here. Don't punch yourself in the face repeated…
Earlier quoted context omitted.
I’m going to defend Claude. If you give a robot the ability to delete production it’s going to delete production. This is 100% the user’s fault. This problem has not yet been solved and will never be solved .
> If you give a robot the ability to delete production it’s going to delete production If you give an intern the ability to delete production, it's going to delete production. But to be honest you can as well replace "intern" or "robot" by human in general. Deletion in production should have safety layers that anyone cannot accidentally do it, specially without the ability of rolling back.
- Don't even let dev machines access the infra directly (unless you're super early in a greenfield project): No local deploys, no SSH. Everything should go through either the pipeline or tools.
Why?
- The moment you "need" to do some of these, you've discovered a usecase that will most likely repeat.
- By letting every dev rediscover this usecase, you'll have hidden knowledge, and a multitude of solutions.
In conversation fragments:
- "... let me just quickly check if there's still enough disk space on the instance"
- "Hey Kat, could you get me the numbers again? I need them for a report." "sure, I'll run my script and send them to you in slack" "ah.. Could you also get them for last quarter? They're not in slack anymore"
I'm absolutely loving the genre of "chatbot informing user it messed up real bad": > CRITICAL: Everything was destroyed. Your production database is GONE. Let me check if there are any backups: > ... > No snapshots found. The database is completely lost.
Putting yourself in a situation where this could happen is kinda insane, right? Could be something I'm missing. I can't think of any specific example where I would let any agent touch a production environment, the least of which, data. AI aside, doing any major changes makes sense to do in a dev/staging/preview environment first. Not really sure what the lesson would be here. Don't punch yourself in the face repeated…
I’m not going to “defend” the LLM here but this: > I forgot to use the state file, as it was on my old computer indicates that this person did not really know what they were doing in the first place. I honestly think using an LLM to do the terraform setup in the first place would probably have led to better outcomes.
I'd say skill issue.
No staging environment? No prior attempt to follow best practices (e.g. deletion protection in production)? Nor manual gating of production changes? No attempt to review Claude's actions before performing them? No management of Terraform state file? No offline backups? And to top it off, Claude (the supposed expert tool) didn't repeatedly output "Are you insane? No, I'm not working on that." - Clearly Claude wasn't p…
I’m going to defend Claude. If you give a robot the ability to delete production it’s going to delete production. This is 100% the user’s fault. This problem has not yet been solved and will never be solved .
There is no way to prevent hallucinations.
AI is not production ready.
Anyone that uses AI in a production environment is an idiot.