I just can’t believe we’re still paying engineers to be Claude wrappers. This is ending soon, right?
Nope, because as I’ve seen first hand most people are too incompetent to use an LLM effectively, as easy as it is.
AI handles incidents, engineers lose touch with their systems
231–240 of 378 posts
Re: AI handles incidents, engineers lose touch with their systems
#232Earlier quoted context omitted.
> 30 minutes of patient, old school troubleshooting This is also the best way to understand a codebase, and it's quite enjoyable. I think the models are getting too egotistical. They're so confident of their fixes they won't bother suggesting basic techniques like isolation of the problem by disabling code paths, for example. They'll keep shotgunning less-and-less likely fixes with undiminishing confidence until the…
> and it's quite enjoyable Used to be somewhat enjoyable. Nothing pleasant about digging around codebase that was heavily affected by the last 12-18 months of AI-ing.
We have a tool at work that was entirely spun out of a prototype that itself was entirely driven by a single guy with an unlimited Claude budget. I only touch it on the edges, where I need to integrate it every so slightly into one of my systems. It's a crucial piece of software for the team, though. I've looked over the source, the thing that strikes me the most is the verbosity. Human software engineers would've been much more terse, so the thing has a fucking lot of code that's doing very little. They're trying to deal with that now, while also driving feature requests forward, and from the distance it looks like an unmanageable shit show.
Re: AI handles incidents, engineers lose touch with their systems
#233I see aviation sentiment raised from time to time. In aviation failure is catastrophic, and systems you operate do not change on the fly (pun intended). You can probably drill SREs that way but you are only teaching them how to _react_ efficiently, not how to _fix_ unique unicorn root causes of these failures. Aviation analogy would be an airplane engineer that tries to drill himself for in-flight engine failures AND…
In aviation all code and doc are available to inspectors when a catastrophic failure or near miss happens.
It's not the case for other industries, and in particular car industry is able to hide about everything software related from inspectors.
Re: AI handles incidents, engineers lose touch with their systems
#234A natural evolution of engineers losing touch with the customers and users. I'm noticing some of the concern play out regarding AI weakening the capabilities of software people. I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patien…
> “A natural evolution of engineers losing touch with the customers and users”
I’m a neophyte to software teams. I work at an accounting consultancy and deal with lots of SaaS platforms. I’m pissing in the wind about this or that product quality issue through the support queue on a weekly basis. There’s prolly an XKCD comic about it.
It’s a rare company that responds to the issues raised through support. Anything that could make engineering more responsive without degrading the product quality would be welcome to us.
Re: AI handles incidents, engineers lose touch with their systems
#235Earlier quoted context omitted.
In my experience at a large bank with unlimited AI, my spend is in the top 5% and I'm leveraging AI just like you. I was in a meeting with someone who had a bug in an application that I don't own myself on Friday so I told claude, "I need you to find this bug the user is experiencing, find out if it's user error or a bug, let the user know and tell the developer what needs to be fixed if needed" 15 minutes later the…
> "I need you to find this bug the user is experiencing, find out if it's user error or a bug, let the user know and tell the developer what needs to be fixed if needed" 15 minutes later the developer ask me if I want it fixed today or Tuesday. Did you find out if it actually fixed the issue? I was on the other end of this last week. Reported an issue to another team, they asked Claude to diagnose, and when they gave…
To be clear, that one shot prompt was supported by nearly a years worth of work around developing an enviroment that allows me that level of effortless automation.
The issues was really simple, I knew the application, I knew what the bug was and already had a workaround in my head. I simply stopped at going down that rabbit hole and told claude to deal.
It identified the issue quickly, found the workaround and made the developer aware. All those things I would have done I just watched it do for me, plus, the app is vibe coded anyways, I just need it fixed and regardless of the AI path it was faster shooting out the bug report. Which, was simply poor processing of a spreadsheet.
Re: AI handles incidents, engineers lose touch with their systems
#236You’d hope these AI incident responders have very constrained production tools to fix things. You’d hope the humans remain familiar with those tools and they are incredibly well documented.
I think the biggest loss is now not knowing if what the agent is claiming was actually done:) but yes, right tools are necessary, adding a boundary will change the position, being out of touch will be recoverable. Otherwise one bad call might be unrecoverable and no amount of familiarity will save you
Re: AI handles incidents, engineers lose touch with their systems
#237Earlier quoted context omitted.
> and it's quite enjoyable Used to be somewhat enjoyable. Nothing pleasant about digging around codebase that was heavily affected by the last 12-18 months of AI-ing.
Depends on the prior state. For one codebase I work in, LLM written code is noticeably and measurably (we have literally measured it for bugs, performance, etc.) better than what existed before. So the LLM code is a welcome change.
I find this shocking though not unbelievable. Could you share how you measure this accurately? I'm interested in integrating such measurements into our services as well.
Re: AI handles incidents, engineers lose touch with their systems
#238Earlier quoted context omitted.
The article you've linked paints it as life-style, entertainment - rich people hobbies, which is different than doing it because it's more productive. People will "trad-code", but not because it's the productive thing to do.
It's fun, and it's serious - keeping heavy horses and bullocks, feed, breeding lines, etc isn't trivial - and they are worked. The serious side is simple enough, local food security - when there's no diesel to seed and harvest with there will be no export crops from here to feed millions across the planet .. but we here will still have smaller crops via animals for our food. I'm literally in one of the most isolated…
In a Mad Max world, the people with guns will come and will take away your stuff. So you better become a war lord and invest in your local army.
I'm sure you've considered that, I'm curious if you could share your thinking on this aspect.
Re: AI handles incidents, engineers lose touch with their systems
#239I see aviation sentiment raised from time to time. In aviation failure is catastrophic, and systems you operate do not change on the fly (pun intended). You can probably drill SREs that way but you are only teaching them how to _react_ efficiently, not how to _fix_ unique unicorn root causes of these failures. Aviation analogy would be an airplane engineer that tries to drill himself for in-flight engine failures AND…
Pilots aren't expected to be design engineers and SREs aren't expected to be software developers.
At least in my experience, operators and on-call engineers aren't necessarily responsible for remediation and validation.
Before you take me too literally, there is, of course, an overlap of skill and trade, but I believe the pilot analogy is more apt than you argue. I do agree that the difference in consequence and stakes is meaningfully important, however.
Re: AI handles incidents, engineers lose touch with their systems
#240Earlier quoted context omitted.
Nah, It's not even good for small code changes. Try using it with Ansible. It spits back complete buffoonery.
I was investigating an ansible playbook yesterday that had a 45 line comment to explain a single apt install command, completely and utterly useless. I am updating my neovim to just collapse all comments, the noise is unbearable.
I've seen people report cleaner code by forbidding agents from writing comments. Anecdotal, but interesting.