AI handles incidents, engineers lose touch with their systems
sylvainkalache.com
AI handles incidents, engineers lose touch with their systems
1–10 of 375 posts
Re: AI handles incidents, engineers lose touch with their systems
#2If capability increase continues as it has, then an incident that cannot be resolved by AI will stump humans no matter the practice.
I like the plane example from the article,but I think in reality it will be like code. 1.5 years ago engineers would routinely say that they still write code by hand here or there to keep their skills sharp, and that's just not something you hear much if at all.
If an SRE is faced with a situation an AI can't solve, then said SRE will use the AI systems to triage further, point it to different places and so on.
This works for SREs with pre-AI experience and intuition, possibly less so with new recruits coming in post-AI. I don't know what the solution to this is, maybe practice drills is it, but I have a hunch the entire field will be subsumed, same as many other engineering fields.
There is only so much need for taste and judgement, before even that has been incorporated into the models.
Re: AI handles incidents, engineers lose touch with their systems
#3Good article and I like the callouts to the aviation industry. For me what's missing is the author should also have touched on CRM and SRM.
Also, that paper "The Ironies of Automation" is one that everyone should read. It's fairly short.
There is a related problem in terms of these situations where the computer system is handing off to the human. It's called "the bumpy transfer of control." Very fascinating concept.
Re: AI handles incidents, engineers lose touch with their systems
#4If capability increase continues as it has, then an incident that cannot be resolved by AI will stump humans no matter the practice. I like the plane example from the article,but I think in reality it will be like code. 1.5 years ago engineers would routinely say that they still write code by hand here or there to keep their skills sharp, and that's just not something you hear much if at all. If an SRE is faced with…
Nah, LLM models are already the new compilers. A commodity only engineers know how to use (in the context of software engineering in production environments)
Re: AI handles incidents, engineers lose touch with their systems
#5The more code writes autonomously, the less intuition the human owners have about that code. Loss of intuition is a seed of technical debt that grows with time. Over a long enough horizon, it can make looking at your own codebase feel like the first day on the job (sometimes at a company you started).
Luckily, there are ways to mitigate this and essentially translate those human intuition of how the codebase “should” be into guardrails for the agents. But without that, your setting your sails in a stochastic sea where each wave looks nothing like the last.
Re: AI handles incidents, engineers lose touch with their systems
#6A natural evolution of engineers losing touch with the customers and users.
I'm noticing some of the concern play out regarding AI weakening the capabilities of software people.
I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patient, old school troubleshooting.
I think what's happening is the AI system draws poorly aligned and led engineers into this ego inflation feedback loop where they are completely detached from reality because these tools can simulate a better one.
Re: AI handles incidents, engineers lose touch with their systems
#7Ironies of Automation is front and center in this article which is awesome. Many of the conversations on AI automation are describing or rediscovering the insights the paper covered.
Re: AI handles incidents, engineers lose touch with their systems
#8A natural evolution of engineers losing touch with the customers and users. I'm noticing some of the concern play out regarding AI weakening the capabilities of software people. I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patien…
I'm seeing this happen in the security space right now. Someone on my team I was helping train and bring along is all of sudden regressing in their understanding of the issues we're working on, and instead focusing on AI tool outputs to do their job for them.
Re: AI handles incidents, engineers lose touch with their systems
#9I find the use of AI like quicksand.
The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed.
Code review is important but it does not replace the mental model I am able to build when I do all the steps of software development manually without AI.
Re: AI handles incidents, engineers lose touch with their systems
#10A natural evolution of engineers losing touch with the customers and users. I'm noticing some of the concern play out regarding AI weakening the capabilities of software people. I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patien…
I it usually doesn't get me in this weird state of mind, but I once spent 6 months (all-in) building a thing that I, once finished, just left alone completely (on disk gathering dust). Weird experience. So I'd say AI physchosis is real.