Live data from Hacker News

AI handles incidents, engineers lose touch with their systems

sylvainkalache.com

121–130 of 379 posts

Re: AI handles incidents, engineers lose touch with their systems

#121

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Ironies_of_Automation https://www.sciencedirect.com/science/article/abs/pii/000510... The article is based on evidence, and states it clearly in the body.

I read the article before commenting, as is my habit. A disagreement is not always a failure to process. To be very explicit, my problem is that the article argues that simulation will be a suitable substitute in the software space, as it is in the aviation space. I see no evidence for that; it's pure hypothesis taken as fact.

The article proposes a resolution based on the challenge and evidence they (correctly) identify at the start.

The simulation solution is backed by their own applied example, and is also the logical intervention based on how the problem is set up.

Perhaps Bainbridge is doing more heavy lifting for me, by filling in blanks and gaps that others are finding.

Re: AI handles incidents, engineers lose touch with their systems

#122
post #70

Earlier quoted context omitted.

Sincerely , I think you're blaming the AI incorrectly there. You just got incompetents on your payroll.

So how do you build competence in a world where AI is preached to be the most reasonable way to solve problems because it's supposed to be faster than humans?

Have a single human AI chef. Everyone else has to write an engineering statement and submit it to the AI chef. All that interaction is outside the codebase. Engineers will take turns - perhaps 1 month stints - being the AI chef.

I guarantee you'll spend less on tokens, have better documentation, better code, and most importantly more competent engineers.

Re: AI handles incidents, engineers lose touch with their systems

#123
post #6

A natural evolution of engineers losing touch with the customers and users. I'm noticing some of the concern play out regarding AI weakening the capabilities of software people. I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patien…

When I hear this I do wonder to myself how they're using AI.

For me, as long as I'm properly RPI looping it and not blindly pressing "yes" then it will nearly always reach the solution, usually a fair bit quicker, because it effectively becomes an ideation machine that can keep more thoughts and knowledge in it than my brain can.

However, I'm using it through giving it the context, it has access to the repos, access to the sytems, I'm telling it where the logs are, I have docs to show it what each part of the system is for.

Along with that, it does depend on my own instincts/knowledge for me to read its response and for me to say "no, you've over-thought/engineered this and this is actually the better solution", but its very are nowadays for me that it can't find the 1 liner, as long as I've fed in the right context, mostly pre-done because I've spent a bit of time building out the context tree for the repos/systems in a way that allows it to know what it needs to know.

I'm not doubting that people spin their wheels and couldn't find the one liner, but if its as simple as you say, that does seem like people who aren't great at LLMs along with a lack of instinct/experience.

I guess in essence, I think you can use LLMs in an old school troubleshooting way, and I find it still speeds that up the majority of the time. Its basically how I use it most of the time. And like old-school troubleshooting, if you build out the LLMs context over time, it also grows in capability, as long as its being used as a tool and not blindly trusted.

Should I not assume that most senior developers who are serious about LLMs do this?

Re: AI handles incidents, engineers lose touch with their systems

#124
post #25

Earlier quoted context omitted.

I'm seeing this happen in the security space right now. Someone on my team I was helping train and bring along is all of sudden regressing in their understanding of the issues we're working on, and instead focusing on AI tool outputs to do their job for them.

I can share a weird story: Usually, I take my time to understand each keyword of the code I'm looking at, especially if it is new to me, like terraform. I work in a team/with one architect, who only did the DevOps/Infra stuff for the past years and I had the expectation he knows what he is doing and talking about. At around 2 weeks, I noticed how his knowledge has severe gaps and how he takes things at face value or…

> It is quite shocking how people went from being humble (learn the basics or close the gaps in understanding) to full on authority on everything and berating people 24/7...

That was never humility. That was fear. This technology is just enabling people to show their through colors. The unearned confidence is a hell of a drug.

Re: AI handles incidents, engineers lose touch with their systems

#126

Author has a good head on their shoulders, but few if any companies are going to spend time on incident simulations for their SREs. Why not? Because even pre-AI, very few companies spend time practicing restoring their backups, or disaster recovery, or picking infrequently-used runbooks to practice, or seeing whether they can easily rotate secrets without downtime, or trying to redploy the system onto another vendor'…

>Why not? Because even pre-AI, very few companies spend time practicing restoring their backups, or disaster recovery, or picking infrequently-used runbooks to practice, or seeing whether they can easily rotate secrets without downtime, or trying to redploy the system onto another vendor's cloud/platform, or, or, or...

Exact. Let insurance cover it, say sorry to your customers twice and shwoop never happened.

Re: AI handles incidents, engineers lose touch with their systems

#127
post #21

Earlier quoted context omitted.

Claude regular spits out six helper functions instead of... A twenty line for loop. It overengineers most things. Overabstracting, deduplicating things that don't need to be. Building metaclasses because it saw a single orchestrator in the whole codebase. If it is a better engineer than you... You need practice.

The funny thing is that if you never understand the codebase then you will keep thinking Claude is doing a great work delivering all this incredible software, when all it has done is created unnecessary tech debt.

Which at some point the developers who actually still do know how to program are either going to have to clean up or in terminal cases rewrite.

In a way all we've done (currently) is drastically expand the amount of technical debt across the whole industry.

Should be profitable for the ones who can still actually program though and haven't let their skills atrophy by letting Claude do everything.

I don't deny there are use cases for LLM's, I just don't buy the hype about them either.

As with all tools, you have to understand how to use them to get done what you need to get done without sticking the chisel through your hand.

Re: AI handles incidents, engineers lose touch with their systems

#128

I am exceedingly tired of poor metaphors that are popping up since AI has taken over writing. No, operating software is not like operating a plane. Not at all in fact. The people operating the software and resolving incidents are the same people who created the software in the first place, and continue to work on it day to day. Pilots have not and don't.

I'm not sure if I agree with the plane analogy either. Aren't planes supposed to have exceptionally stable hardware and software platform? Does that not significantly reduce failure modes? Most production software by comparison is constantly evolving. Unless you design very simple or basic training examples, most of it's going to be out of date soon.

[deleted]

Re: AI handles incidents, engineers lose touch with their systems

#129
post #6

A natural evolution of engineers losing touch with the customers and users. I'm noticing some of the concern play out regarding AI weakening the capabilities of software people. I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patien…

> 30 minutes of patient, old school troubleshooting

This is also the best way to understand a codebase, and it's quite enjoyable.

I think the models are getting too egotistical. They're so confident of their fixes they won't bother suggesting basic techniques like isolation of the problem by disabling code paths, for example. They'll keep shotgunning less-and-less likely fixes with undiminishing confidence until the code is full of fixes that do nothing. Then they'll double down on why these should be kept.

Re: AI handles incidents, engineers lose touch with their systems

#130

Earlier quoted context omitted.

I read the article before commenting, as is my habit. A disagreement is not always a failure to process. To be very explicit, my problem is that the article argues that simulation will be a suitable substitute in the software space, as it is in the aviation space. I see no evidence for that; it's pure hypothesis taken as fact.

The article proposes a resolution based on the challenge and evidence they (correctly) identify at the start. The simulation solution is backed by their own applied example, and is also the logical intervention based on how the problem is set up. Perhaps Bainbridge is doing more heavy lifting for me, by filling in blanks and gaps that others are finding.

I would say that Bainbridge supports my position, if anything. Take a look at section 2.3 of her paper, where she lists all the reasons why simulation can be unsuitable in preparing for complex incidents: https://davidjusth.com/s/Ironies-of-Automation_Bainbridge_19...
Post reply on HN