Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI handles incidents, engineers lose touch with their systems

sylvainkalache.com

Post reply on HN
Print view View original

111–120 of 379 posts

Re: AI handles incidents, engineers lose touch with their systems

#111

Post by duskdozer »

Traubenfuchs wrote: post #58
hypfer wrote: post #29

Meta: The blinking cursor of the "logo" of the blog being sticky in the top left corner makes it impossible for me to read the text. It constantly fires interrupts at me. Depending on what your goals as the author are, you may or may not want that. Being able to scroll it out of view might be enough to achieve the aesthetics goal, and the goal of people actually listening to you.

Not just you. Must be nice being fully neurotypical and „not seeing“ all of this kind of stuff.

The internet for me (and increasingly non-internet programs) is unusable at this point without excessive modifications. It's hard to imagine what it's like being unbothered by it all.

Re: AI handles incidents, engineers lose touch with their systems

#112

Post by solatic »

Author has a good head on their shoulders, but few if any companies are going to spend time on incident simulations for their SREs.

Why not? Because even pre-AI, very few companies spend time practicing restoring their backups, or disaster recovery, or picking infrequently-used runbooks to practice, or seeing whether they can easily rotate secrets without downtime, or trying to redploy the system onto another vendor's cloud/platform, or, or, or... It is the least-sexy operations work that exists. No executive cares about this. Ops organizations push for flashy work, same as everybody else: new infrastructure for new projects, cool chatbots, new flashy dashboards, make charts go up and to the right, etc.

Airline pilots go through disaster simulation training because the government mandates that training. If it wasn't a condition of holding a pilot's license, no company would pay for it.

Want SREs to spend time training for disasters? Take a step back. Support professional licensure. Make it a condition of holding a license. You won't get industry-wide professional behavior until you professionalize the work. It won't happen without licensing because every corner cut that is not immediately visible to consumers translates to additional profit, and increasing competition eventually requires these corners to be cut in order to keep up with competition and stay in business. Forcing all players to submit to licensing requires all players to pay these costs and thus forbids them from cutting them to become more competitive.

Re: AI handles incidents, engineers lose touch with their systems

#113

Post by Tade0 »

danielheath wrote: post #65
wccrawford wrote: post #53

Earlier quoted context omitted.

I don't think it's generalizable. The kind of person who copy and pastes from the AI is the kind who did the same from StackOverflow before. It's more compelling, and we probably see more of them because of it, but it's the same general thing. The kind of person who insists on understanding things and working through the problem has always been rarer. It's not "humble", it's "inquisitive" and "persistent".

I'm seeing people who _used to be_ like that losing that understanding without realizing it's happening - they have a superficial idea of what the code is doing, enough to feel like they understand it, but the change is apparent when watching them handle something unexpected.

LLMs reduced interest on tech debt, but it's still there and people who have a tendency to acquire it will go bankrupt eventually.

Re: AI handles incidents, engineers lose touch with their systems

#114

Post by shakna »

gnz11 wrote: post #99
jjav wrote: post #63

Earlier quoted context omitted.

> A twenty line for loop. It overengineers most things. Anecdote I like to tell.. I was working on a financial planning software, intentionally purely vibe coded as an experiment. I eventually discovered AI had implemented seven duplicate copies of tax calculation functions. All of them different. All of them wrong. All of them giving different answers for same input. Not even the most junior of newbie junior enginee…

Which model though? I have similar anecdotes but all with older models. The jump in capabilities in the last 6 months has been substantial.

I have literally been hearing that, over and over and over, since '22.

And whilst it is obvious things are growing... Saying that, sounds almost entirely like the person saying it cannot objectively look at the environment. If everything has changed in the last six months, why has the industry not radically changed to match it?

Everything really did change with the Pentium II. It did with 3dfx. It did with Damerau's taken on Levenshtein. Hell, everything changed with React. The AI leap with seq2seq completely revolutionised the entire industry. But... Its kid, the LLM? Really?

Re: AI handles incidents, engineers lose touch with their systems

#115

Post by intended »

wccrawford wrote: post #53
mawadev wrote: post #25

Earlier quoted context omitted.

I can share a weird story: Usually, I take my time to understand each keyword of the code I'm looking at, especially if it is new to me, like terraform. I work in a team/with one architect, who only did the DevOps/Infra stuff for the past years and I had the expectation he knows what he is doing and talking about. At around 2 weeks, I noticed how his knowledge has severe gaps and how he takes things at face value or…

I don't think it's generalizable. The kind of person who copy and pastes from the AI is the kind who did the same from StackOverflow before. It's more compelling, and we probably see more of them because of it, but it's the same general thing. The kind of person who insists on understanding things and working through the problem has always been rarer. It's not "humble", it's "inquisitive" and "persistent".

The difference between the two is that friction is learning.

the less friction the less growth.

So even if they copied from SO they would have better knowledge than an Ai user.

This isn’t speculation either, theres an MIT study which this is based on. https://www.media.mit.edu/publications/your-brain-on-chatgpt...

Re: AI handles incidents, engineers lose touch with their systems

#116

Post by gnfargbl »

intended wrote: post #109
gnfargbl wrote: post #103

This article leads up to a hypothesis presented without evidence. Yes, it's clear that if AI agents handle routine issues then only the most complex issues will be handled by humans. In no way does it follow that humans will be less prepared to handle those incidents than they are now; by definition, this class of problems have no rote solutions. Each issue of this type already requires deep system knowledge to remed…

https://en.wikipedia.org/wiki/Ironies_of_Automation https://www.sciencedirect.com/science/article/abs/pii/000510... The article is based on evidence, and states it clearly in the body.

I read the article before commenting, as is my habit. A disagreement is not always a failure to process.

To be very explicit, my problem is that the article argues that simulation will be a suitable substitute in the software space, as it is in the aviation space. I see no evidence for that; it's pure hypothesis taken as fact.

Re: AI handles incidents, engineers lose touch with their systems

#117

Post by neuronic »

konschubert wrote: post #61
bob1029 wrote: post #6

A natural evolution of engineers losing touch with the customers and users. I'm noticing some of the concern play out regarding AI weakening the capabilities of software people. I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patien…

It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself. EDIT: don’t get me wrong. I still think AI is incredibly useful for a lot of tasks! But when implementing an architecturally hairy thing, I find it less stressful and equally quick to jump down…

Your assumption is that LLMs will ever leave this uncanny valley.

Maybe unforeseen breakthroughs and different architectures are achieved. Given LLM fundamental shortcomings grounded in mathematics and information theory, I highly doubt they will and we will always need to deal with these issues in some capacity.

Re: AI handles incidents, engineers lose touch with their systems

#118

Post by neuronic »

konschubert wrote: post #56
krtkush wrote: post #9

I find the use of AI like quicksand. The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed. Code review is important but it does not replace the mental model I am able to build when I do all the steps of software development manually without AI.

It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.

You don’t need to copy-paste the same comment in every reply. It seems like agenda pushing more than contributing.

Re: AI handles incidents, engineers lose touch with their systems

#119

Post by bsoqk »

krtkush wrote: post #9

I find the use of AI like quicksand. The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed. Code review is important but it does not replace the mental model I am able to build when I do all the steps of software development manually without AI.

>The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed.

Most of us build a system to deliver a product to make money. We don't feel empty for not knowing how the system works, since we don't really care: it's just a means to an end.

Re: AI handles incidents, engineers lose touch with their systems

#120

Post by huurtehoog »

noir_lord wrote: post #108
konschubert wrote: post #61

Earlier quoted context omitted.

It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself. EDIT: don’t get me wrong. I still think AI is incredibly useful for a lot of tasks! But when implementing an architecturally hairy thing, I find it less stressful and equally quick to jump down…

Pretty much, The one thing I use it for is as a sanity check, pretty much "Look at , point out issues you see, summarise them tersely" and it'll spot stuff a code review by a human might have spotted (in the mythical land where people actually do code reviews properly and don't just flag a spelling mistake to "show they looked at it"). Beyond that I don't trust it at all and I still write all my code the meat sack wa…

If anything, I think this hype cycle is fast exposing just how many people, teams, and companies just don't care about what is correct. They just wanna feel good about themselves and get paid.

I for one welcome the fact that this whole thing has driven me back to books and deeper into the fundamentals. I have never read so much on math, hardware, and history as in the past 3 years or so.

Post reply on HN