The threat is comfortable drift toward not understanding what you're doing • Hacker News Board

Re: The threat is comfortable drift toward not understanding what you're doing

#281

Post by anthk » Sun, Apr 05, 2026, 2:27 PM UTC

II2II wrote: post #258

lukev wrote: post #179

Earlier quoted context omitted.

I think that's too easy an analogy, though. Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not. As someone who uses LLMs all day for coding, and who regularly bumps against the boundaries of what they're capable of, that's very much not the case. The only reason I can use them effectively is because I know what good…

> Calculators are deterministically correct Calculators are deterministic, but they are not necessarily correct. Consider 32-bit integer arithmetic: 30000000 * 1000 / 1000 30000000 / 1000 * 1000 Mathematically, they are identical. Computationally, the results are deterministic. On the other hand, the computer will produce different results. There are many other cases where the expected result is different from what a…

Good languages with proper number towers will deal with both cases in equal terms.

Re: The threat is comfortable drift toward not understanding what you're doing

#282

Post by lxgr » Sun, Apr 05, 2026, 2:29 PM UTC

qsera wrote: post #133

samusiam wrote: post #116

Earlier quoted context omitted.

> Aren't they currently propped up by investor money? Are Chinese model shops propped up by investor money? Is Google? Open weights models are only 6 months behind SOTA. If new model development suddenly stopped, and today's SOTA models suddenly disappeared, we would still have access to capable agents.

>we would still have access to capable agents. But they would be outdated, right? Would an agent that can only code in COBOL would be as useful today?

> But they would be outdated, right?

Outdated compared to what? In your counterfactual, VC funded agents don't exist anymore, no?

Your argument, if I understand it correctly, is that they might somehow go away entirely when VC funding dries up, when more realistically they'll probably at most become twice as expensive or regress half a year in performance.

Re: The threat is comfortable drift toward not understanding what you're doing

#283

Post by FrojoS » Sun, Apr 05, 2026, 2:29 PM UTC

ipaddr wrote: post #274

FrojoS wrote: post #165

Every PhD program I'm aware of has a final hurdle known as the defence. You have to present your thesis while standing in front of a committee, and often the local community and public. They will asks questions and too many "I don't know" or false answers would make you fail. So, there is already a system in place that should stop Bob from graduating if he indeed learned much less than Alice. A similar argument can b…

In the future the llms can answer those questions for you by listening and feeding you answers into your headset. The process you describe is a gate keeping exercise which will change to include llm judges at somepoint.

That would be cheating. If the exam is 'gate keeping', I will say that it is a gate worth keeping.

To be clear, I am not against alternative forms of education. Degrees are optional. But if you want a degree, there have to be exams and cheating has to be prevented.

Re: The threat is comfortable drift toward not understanding what you're doing

#284

Post by jurgenburgen » Sun, Apr 05, 2026, 2:29 PM UTC

QuantumNomad_ wrote: post #57

sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> if Bob can do things with agents, he can do things I’ve been reminded lately of a conversation I had with a guy at hacker space cafe around ten years ago in Berlin. He had been working as a programmer for a significantly longer time than me. Long enough that for many years of his career, he had been programming in assembly. He was lamenting that these days, software was written in higher level languages, and that m…

The difference is that you don’t need to review the machine code produced by a compiler.

The same is not true for LLM output. I can’t tell my manager I don’t know how to fix something in production the agent wrote. The equivalent analogy would be if we had to know both the high-level language _and_ assembly.

Re: The threat is comfortable drift toward not understanding what you're doing

#285

Post by sumeno » Sun, Apr 05, 2026, 2:30 PM UTC

ahussain wrote: post #280

> When his supervisor sent him a paper to read, Bob asked the agent to summarize it. When he needed to understand a new statistical method, he asked the agent to explain it. When his Python code broke, the agent debugged it. When the agent's fix introduced a new bug, it debugged that too. When it came time to write the paper, the agent wrote it. Bob's weekly updates to his supervisor were indistinguishable from Alice…

Faster doesn't always mean better. I've "learned" things from LLM really fast, but I don't retain the information the same way as if I had taken my time to really work through it

Re: The threat is comfortable drift toward not understanding what you're doing

#286

Post by bigfishrunning » Sun, Apr 05, 2026, 2:30 PM UTC

wiseowise wrote: post #256

bigfishrunning wrote: post #205

Earlier quoted context omitted.

People never cared about delivering Alices; they were an implementation detail. I think the article argues that they're still an important one, but one that isn't produced automatically anymore

The article is talking about science research in the context of astrophysics, not coding sweatshops.

I was also talking about producing researchers for academia.

Re: The threat is comfortable drift toward not understanding what you're doing

#287

Post by edbmiller69 » Sun, Apr 05, 2026, 2:31 PM UTC

sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

No - you need to understand the details in order to do the “high level” work.

Re: The threat is comfortable drift toward not understanding what you're doing

#288

Post by michaelcampbell » Sun, Apr 05, 2026, 2:32 PM UTC

sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue.

I am in the same boat, but close enough to retirement that I'm less "scared" about it. For me I'm moving up the chain; not people management, but devoting a lot more of my time up the abstraction continuum. Looking a lot more at overall designs and code quality and managing specs and inputs and requirements.

I wrote some design docs past few days for a big project the team is embarking on. We never had that before, at least not in the level of detail (per time quantum) that I was able to produce. Used 2 models from 2 companies - one to write, one to review, and bounce between them until the 3 of us agree.

Honestly it didn't take any less time than I would have done it alone, but the level of detail was better, and covered more edge cases. Calling it a "win" right now. I still enjoy it, as most of the code I/we was/are writing is mostly fancy CRUD anyway, and doesn't have huge scaling problems to solve (and too few devs I feel are being honest about their work, here).

Re: The threat is comfortable drift toward not understanding what you're doing

#289

Post by ozim » Sun, Apr 05, 2026, 2:32 PM UTC

roenxi wrote: post #40

djaro wrote: post #14

Earlier quoted context omitted.

> So if Bob can do things with agents, he can do things. The problem arrises when Bob encounters a problem too complex or unique for agents to solve. To me, it seems a bit like the difference between learning how to cook versus buying microwave dinners. Sure, a good microwave dinner can taste really good, and it will be a lot better than what a beginning cook will make. But imagine aspiring cooks just buying premade…

That doesn't sound like much of an issue. Bob was already going to encounter problems that are too large and complex for him to solve, agents or otherwise. Life throws us hard problems. I don't recall if we even assumed Bob was unusually capable, he might be one of life's flounderers. I'd give good odds that if he got through a program with the help of agents he'll get through life achieving at least a normal level o…

Market values bulldozers for bulldozing jobs. No one is going to use bulldozers to mow a lawn.

If Bob is going to spend $500 in tokens for something I can do for $50.

I think Bob is not going to stay long in lawn mowing market driving a bulldozer.

Re: The threat is comfortable drift toward not understanding what you're doing

#290

Post by steveBK123 » Sun, Apr 05, 2026, 2:32 PM UTC

ipaddr wrote: post #263

steveBK123 wrote: post #211

For the people arguing that the output is the code and the faster we generate it the better.. I do wonder where all the novel products produced by 10x devs who are now 100x with LLMs, the “idea guys” who can now produce products from whole clothe without having to hire pesky engineers.. where is the one-man 10 billion dollar startups, etc? We are 3-4 years into this mania and all I see on the other end of it is the L…

Marketing is the moat llms haven't been able to overcome. Being able to create a Word clone is easier but the difficulty of selling it is as hard or harder than ever. Show me an llm that can sell my product and find market fit. In reality llms are taking away profitable tools and keeping the revenue themselves.

Right Very Rory Sutherland kind of thought - marketing doesn’t make sense. It is alchemy.

If I told you the drink tastes bad, is an off putting color, comes in a small bottle, and is expensive you wouldn’t believe it would work. But Red Bull made billions.