The threat is comfortable drift toward not understanding what you're doing • Hacker News Board

Re: The threat is comfortable drift toward not understanding what you're doing

#261

Post by jappgar » Sun, Apr 05, 2026, 2:15 PM UTC

mkovach wrote: post #240

This isn't new. It's been the same problem for decades, not what gets built, but what gets accepted. Weak ownership, unclear direction, and "sure, I guess" reviews were survivable when output was slow. When changes came in one at a time, you could get away with not really deciding. AI doesn't introduce a new failure mode. It puts pressure on the old one. The trickle becomes a firehose, and suddenly every gap is visib…

This is exactly the problem I see today.

And it's not just a volume problem.

Mediocre devs previously couldn't complete a project by themselves and were forced to solicit help and receive feedback along the way.

When all managers care about is "shipping", development becomes a race to the bottom. Devs who used to collaborate are now competing. Whoever gets the slop into the codebase fastest, wins.

Re: The threat is comfortable drift toward not understanding what you're doing

#262

Post by lxgr » Sun, Apr 05, 2026, 2:15 PM UTC

> for someone who doesn't yet have that intuition, the grunt work is the work

Very well said. I think people are about to realize how incredibly fortunate and exceptional it is to actually get paid, and in our industry very well, through a significant fraction of one's career while still "just" doing the grunt work, that arguably benefits the person doing it at least as much as the employer.

A stable paid demand for "first-year grad student level work" or the equivalent for a given industry is probably not the only possible way to maintain a steady supply of experts (there's always the option of immense amounts of student debt or public funding, after all), but it sure seems like a load-bearing one in so many industries and professions.

At the very least, such work being directly paid has the immense advantage of making artificially (often without any bad intentions!) created bullshit tasks that don't exercise actually relevant skillsets, or exercise the wrong ones, much easier to spot.

Re: The threat is comfortable drift toward not understanding what you're doing

#263

Post by ipaddr » Sun, Apr 05, 2026, 2:15 PM UTC

steveBK123 wrote: post #211

For the people arguing that the output is the code and the faster we generate it the better.. I do wonder where all the novel products produced by 10x devs who are now 100x with LLMs, the “idea guys” who can now produce products from whole clothe without having to hire pesky engineers.. where is the one-man 10 billion dollar startups, etc? We are 3-4 years into this mania and all I see on the other end of it is the L…

Marketing is the moat llms haven't been able to overcome. Being able to create a Word clone is easier but the difficulty of selling it is as hard or harder than ever.

Show me an llm that can sell my product and find market fit.

In reality llms are taking away profitable tools and keeping the revenue themselves.

Re: The threat is comfortable drift toward not understanding what you're doing

#264

Post by bitwize » Sun, Apr 05, 2026, 2:15 PM UTC

djaro wrote: post #14

sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> So if Bob can do things with agents, he can do things. The problem arrises when Bob encounters a problem too complex or unique for agents to solve. To me, it seems a bit like the difference between learning how to cook versus buying microwave dinners. Sure, a good microwave dinner can taste really good, and it will be a lot better than what a beginning cook will make. But imagine aspiring cooks just buying premade…

Bob+agents is going to be able to solve much more complex problems than Bob without agents.

That's the true AI revolution: not the things it can accelerate, the things it can put in reach that you wouldn't countenance doing before.

Re: The threat is comfortable drift toward not understanding what you're doing

#265

Post by alex_suzuki » Sun, Apr 05, 2026, 2:16 PM UTC

lelanthran wrote: post #131

mattmanser wrote: post #62

Earlier quoted context omitted.

The authors point went a little over your head. It doesn't matter if Bob can be normal. There was no point to him being paid to be on the program. From the article: If you hand that process to a machine, you haven't accelerated science. You've removed the only part of it that anyone actually needed.

> It doesn't matter if Bob can be normal. There was no point to him being paid to be on the program. Yeah, I'm surprised at the number of people who read the article and came away with the conclusion that the program was designed to churn deliverables, and then they conclude that it doesn't matter if Bob can only function with an AI holding his hand, because he can still deliver. That isn't the output of the program;…

It’s a fairly long article, maybe they had it summarized and came to that conclusion…

Re: The threat is comfortable drift toward not understanding what you're doing

#266

Post by zozbot234 » Sun, Apr 05, 2026, 2:16 PM UTC

AlexWilkins12 wrote: post #72

Ironically, this article reeks of AI-generated phrases. Lot's of "It's not X, it's Y". eg: - "The failure mode isn't malice. It's convenience", - "You haven't saved time. You've forfeited the experience that the time was supposed to give you." - "But the real threat isn't either of those things. It's quieter, and more boring, and therefore more dangerous. The real threat is a slow, comfortable drift toward not unders…

Yes, the overwrought "It's not X it's Y" are signals of LLM involvement. No human uses them like that all the frickin' time. AI loves this construct way too much and cannot really tell whether the contrast is relevant/actually makes sense.

Re: The threat is comfortable drift toward not understanding what you're doing

#267

Post by sunir » Sun, Apr 05, 2026, 2:17 PM UTC

I think the mountain of things I don’t understand was already huge. It doesn’t stop me from getting a grip over the things I need to be responsible for and using tools to contain complexity irrelevant to me. Like many scientists have a stats person.

The risk is that civilization is over its skis because humans are lazy. Humans are always lazy. In science there’s a limit to bs because dependent works fail. In economics there’s a crash. In physics stuff breaks. Then there is a correction.

Re: The threat is comfortable drift toward not understanding what you're doing

#268

Post by wongarsu » Sun, Apr 05, 2026, 2:18 PM UTC

II2II wrote: post #258

lukev wrote: post #179

Earlier quoted context omitted.

I think that's too easy an analogy, though. Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not. As someone who uses LLMs all day for coding, and who regularly bumps against the boundaries of what they're capable of, that's very much not the case. The only reason I can use them effectively is because I know what good…

> Calculators are deterministically correct Calculators are deterministic, but they are not necessarily correct. Consider 32-bit integer arithmetic: 30000000 * 1000 / 1000 30000000 / 1000 * 1000 Mathematically, they are identical. Computationally, the results are deterministic. On the other hand, the computer will produce different results. There are many other cases where the expected result is different from what a…

A good calculator will however do this correctly (as in: the way anyone would expect). Small cheap calculators revert to confusing syntax, but if you pay $30 for a decent handheld calculator or use something decent like wolframalpha on your phone/laptop/desktop you won't run into precision issues for reasonable numbers.

Re: The threat is comfortable drift toward not understanding what you're doing

#269

Post by threatofrain » Sun, Apr 05, 2026, 2:19 PM UTC

lukev wrote: post #250

yunwal wrote: post #230

Earlier quoted context omitted.

Determinism just means you don't have to use statistics to approach the right answer. It's not some silver bullet that magically makes things understandable and it's not true that if it's missing from a system you can't possibly understand it.

That's not what I mean. If I use a calculator to find a logarithm, and I know what a logarithm is, then the answer the calculator gives me is perfectly useful and 100% substitutable for what I would have found if I'd calculated the logarithm myself. If I use Claude to "build a login page", it will definitely build me a login page. But there's a very real chance that what it generated contains a security issue. If I'm…

Those two tasks are just very different. In one world you have provided a complete specification, such as 1 + 1, for which the calculator responds with some answer and both you and the machine have a decidable procedure for judging answers. In another world you have engaged in a declaration for which the are many right and wrong answers, and thus even the boundaries of error are in question.

It's equivalent to asking your friend to pick you up, and they arrive in a big vs small car. Maybe you needed a big car because you were going to move furniture, or maybe you don't care, oops either way.

Re: The threat is comfortable drift toward not understanding what you're doing

#270

Post by lxgr » Sun, Apr 05, 2026, 2:20 PM UTC

vrganj wrote: post #197

cfloyd wrote: post #194

Earlier quoted context omitted.

I held this point of view for a while but I came to the (possibly naive) conclusion that it was just forced self-assurance. Truth is, the issues with sub-par output are just a prompting and supervision deficiency. An agent team can produce better end product if supervised and promoted correctly. The issue is most don’t take the time to do that. I’m not saying I like that this is true, quite the opposite. It is the re…

At some point the herding of idiot savants becomes more work than just doing the damn thing yourself in the first place.

I'm happy to herd idiots all my life if they come out of it smarter than they went in. The real tragedy with current LLM agents is that they're effectively stateless, and so all the effort of "educating" them feels wasted.

Once continuous learning is solved, I predict the problem addressed by TFA to become orders of magnitude bigger: What's the motivation for anyone to teach a person if an LLM can learn it much faster, will work for you forever, and won't take any sick days or consider changing careers?