Live data from Hacker News

AI handles incidents, engineers lose touch with their systems

sylvainkalache.com

301–310 of 378 posts

Re: AI handles incidents, engineers lose touch with their systems

#301
post #280

Wasn't that the idea? Withe cars on roads, didn't we literally lose the touch with the ground? I guess the desire was to "lose" touch with all dirty and hard work areas.

That's what the linked article is talking about. Training for extreme failure.

Sure you have self driving cars and everything that make you more and more disconnected, but still highways more dangerous to drive whenever there is a storm or ice on the road. For the same reason, driving tests are harder in the north.

Re: AI handles incidents, engineers lose touch with their systems

#302
The way forward is clear. Improve the AI such that the hardest incidents can be handled by AI alone. Whether we want this or not is independent of the trendline which points to this future regardless.

Imagine a future where humanity understands none of the underlying technology and AI just babysits us on a daily basis. Wall-e.

Programming will devolve into unintelligible anti patterns and will be written in convoluted languages humanity doesn’t understand. But none of this matters because AI will be handling all of it.

Re: AI handles incidents, engineers lose touch with their systems

#303
post #9

I find the use of AI like quicksand. The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed. Code review is important but it does not replace the mental model I am able to build when I do all the steps of software development manually without AI.

It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.

I doubt AI will be good enough to _scale_ many things because human language isn't sufficient to explain it (especially when not understanding what is being built). Humans invented more concise languages (math, C++) to pair with their intellectual acuity. [1]

In one extreme, we have art, which cannot be accurately reproduced or explained in any language (unless a concise language was used to create it, like math or CS art).

At the other end: 1 + 1 = 2 and PI = 3.14...

[1] It seems any language that has a binary outcome (correct/wrong) helps the AI tremendously, however when e.g. creating software, that software is not functioning in isolation. The software is an attempt to codify a fuzzy human system or need, and that information only flows in one direction: human to machine.

Re: AI handles incidents, engineers lose touch with their systems

#304

Earlier quoted context omitted.

> 30 minutes of patient, old school troubleshooting This is also the best way to understand a codebase, and it's quite enjoyable. I think the models are getting too egotistical. They're so confident of their fixes they won't bother suggesting basic techniques like isolation of the problem by disabling code paths, for example. They'll keep shotgunning less-and-less likely fixes with undiminishing confidence until the…

> and it's quite enjoyable Used to be somewhat enjoyable. Nothing pleasant about digging around codebase that was heavily affected by the last 12-18 months of AI-ing.

The field is already exploding in SLOC. If LLMs can generate 10x but only produce a 2x productivity boost in reasoning or understanding - each year -then neither humans or AI will be able to keep up with the mountains of slop that power everything from websites, telecom, banking, automotive, aerospace, industrial, medical, and renewables.

Guess we’ll need even more human software engineers to fix it.

Re: AI handles incidents, engineers lose touch with their systems

#305

Earlier quoted context omitted.

We have a tool at work that was entirely spun out of a prototype that itself was entirely driven by a single guy with an unlimited Claude budget. I only touch it on the edges, where I need to integrate it every so slightly into one of my systems. It's a crucial piece of software for the team, though. I've looked over the source, the thing that strikes me the most is the verbosity. Human software engineers would've be…

I sometimes wonder if the verbosity gives the LLM more “margin for error” when generating text that passes requirements.

10 lines of checks per line of actual execution. Anxiety for their stable pay

Re: AI handles incidents, engineers lose touch with their systems

#306
post #6

A natural evolution of engineers losing touch with the customers and users. I'm noticing some of the concern play out regarding AI weakening the capabilities of software people. I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patien…

> A natural evolution of engineers losing touch with the customers and users.

I disagree. I think this happens as soon as the MBAs come on board, where everything becomes a metric, and you work towards OKRs.

Re: AI handles incidents, engineers lose touch with their systems

#307

Earlier quoted context omitted.

We have a tool at work that was entirely spun out of a prototype that itself was entirely driven by a single guy with an unlimited Claude budget. I only touch it on the edges, where I need to integrate it every so slightly into one of my systems. It's a crucial piece of software for the team, though. I've looked over the source, the thing that strikes me the most is the verbosity. Human software engineers would've be…

I sometimes wonder if the verbosity gives the LLM more “margin for error” when generating text that passes requirements.

It's simpler. The verbosity, the line of code count, is what has gotten people excited about it. So everyone optimizes for verbosity.

Re: AI handles incidents, engineers lose touch with their systems

#308

Earlier quoted context omitted.

Even that would be acceptable. But they still blame the programmer, who was pressured into this, for any fallouts because of it. Basically trying to have it both ways. The unreasonableness of it is infuriating.

Pretty much this, they want to impose one method, but they don’t want to own the mistakes that it creates. Sometimes it felt like the bullies that are saying “why are you making me hurting you?”.

I see it as laying the groundwork for firing. This is bound to blow-up in production eventually which would be a good enough excuse. If it doesn't, reduce token budget and squeeze some more until it breaks.

Re: AI handles incidents, engineers lose touch with their systems

#309

Earlier quoted context omitted.

I've seen people report cleaner code by forbidding agents from writing comments. Anecdotal, but interesting.

Mozilla's Firefox AGENTS.md begins with: Limit the amount of comments you put in the code to a strict minimum. You should almost never add comments, except sometimes on non-trivial code, function definitions if the arguments aren't self-explanatory, and class definitions and their members. Do not remove existing comments unless they are directly related to what you are changing. https://searchfox.org/firefox-main/sou…

Something like this seems unlikely to work for a behavior so burned into them. It'd be better to do a second pass to delete all the nonsense comments.

Re: AI handles incidents, engineers lose touch with their systems

#310
post #229

Earlier quoted context omitted.

It also documents itself extensively

But then you’ll use it to change something, and it won’t update any of the extensive documentation. So then it’s filled with lies.

Sounds like human generated code and docs. TBH
Post reply on HN