The threat is comfortable drift toward not understanding what you're doing • Hacker News Board

Re: The threat is comfortable drift toward not understanding what you're doing

#561

Post by bsder » Sun, Apr 05, 2026, 11:38 PM UTC

lukev wrote: post #179

NiloCK wrote: post #162

Earlier quoted context omitted.

People would have said the same about graphing calculators or calculators before that. Socrates said the same thing about the written word. The determining factor is always "did I come up with this tool". Somehow, subsequent generations always manage to find their own competencies (which, to be fair, may be different). This isn't guaranteed to play out, but it should be the default expectation until we actually see g…

I think that's too easy an analogy, though. Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not. As someone who uses LLMs all day for coding, and who regularly bumps against the boundaries of what they're capable of, that's very much not the case. The only reason I can use them effectively is because I know what good…

> Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not.

That's not actually true. The HP-12C calculator is still the dominant calculator in business schools 45 years later precisely because it did take expert judgement to determine whether certain interest and amortization calculations were reasonable.

Re: The threat is comfortable drift toward not understanding what you're doing

#562

Post by matheusmoreira » Sun, Apr 05, 2026, 11:40 PM UTC

krackers wrote: post #554

matheusmoreira wrote: post #546

Earlier quoted context omitted.

I did implement it though. I wrote the code myself. Claude explained to me how all of those things worked, showed me code and examples to illustrate, walked me through the algorithms step by step. It turned out to not be as insurmountably complex as I thought it was. I'm also manually writing articles about all of those things to crystallize everything I learned. Claude has been amazing for code review. Having someon…

Oh ok that's different! From your comment I assumed that it was Claude Code doing the implementation while you only gave it high level concepts (e.g. "add a generational GC"). But if you use it as a resource to clarify concepts while you write the implementation yourself, then it's no different than having a tutor who helps you understand so you can do the homework yourself. Given the article, I don't think most peop…

Yeah. I've also been using it as a manager of sorts. I'm not a very organized person. My mental OS can't schedule things properly and I have trouble starting tasks. Claude's been extremely helpful by giving me structure and direction. Just asking Claude to scour the project's chats and give me a list of steps I can easily and immediately start executing pretty much cured me of my ADHD. I frequently ask Opus to do full codebase reviews just so I can have it do this for me.

I totally made Claude do my own homework on my static site generator project though. It's even added itself as coauthor. Haven't even read the commits yet. It was just ripping out broken code and eliminating dependencies. Nothing that even approaches the complexity of my programming language work.

Re: The threat is comfortable drift toward not understanding what you're doing

#563

Post by greedo » Sun, Apr 05, 2026, 11:40 PM UTC

carlosjobim wrote: post #503

mattmanser wrote: post #492

Earlier quoted context omitted.

Seriously? The article is about scientists learning to do science, not programming. I know we're not supposed to say RTFA, but your comment really takes the cake.

My mistake then. There's so much AI discussion here on HN from the developer perspective, that it's easy to get lost. But why wouldn't my comment apply to all science? You have to make experiments and gather evidence, otherwise it's not science, it's just academia.

If you read the article, it covers much of this. It's not just gathering evidence, but evaluating the data you've collected.

Re: The threat is comfortable drift toward not understanding what you're doing

#564

Post by [deleted] » Sun, Apr 05, 2026, 11:42 PM UTC

HauntingPin wrote: post #306

sumeno wrote: post #189

Earlier quoted context omitted.

That's what the program he just took was supposed to be for, learning not output. You've just reinvented the article from first principles, congrats

Sometimes I wonder how deeply some people actually read these articles. What's the point of the comments if all we're doing is re-explaining what's already explained in such a precise and succint manner? It's a fantastic article. It's so well-written and clear. And yet we're stuck going in a circle repeating what's in the article to people who either didn't read it, or didn't read it with the care it deserves.

[deleted]

Re: The threat is comfortable drift toward not understanding what you're doing

#565

Post by Yokohiii » Sun, Apr 05, 2026, 11:47 PM UTC

julenx wrote: post #496

Yokohiii wrote: post #319

Earlier quoted context omitted.

This is my primary personal concern. I think it could be an silent psychological landmine going off way too late (sic). In a living codebase you spent long stretches to learn how it works. It's like reading a book that doesn't match your taste, but you eventually need to understand and edit it, so you push through. That process is extremely valuable, you will get familiar with the codebase, you map it out in your hea…

I believe the detachment gets exacerbated by the fact that others are simultaneously modifying the codebase at speeds that doesn't allow you to keep up. Depending on how the codebase boundaries and ownership are defined, this directly impacts your ability to reason about the whole system and therefore influence direction.

And you can't ask anyone about it, because they are also detached.

Re: The threat is comfortable drift toward not understanding what you're doing

#566

Post by throwaway27448 » Sun, Apr 05, 2026, 11:50 PM UTC

mrob wrote: post #399

Wowfunhappy wrote: post #329

Earlier quoted context omitted.

Then the social paradigm needs to change. Is everyone just going to roll over and die while AI destroys academia (and possibly a lot more)? Last September, Tyler Austin Harper published a piece for The Atlantic on how he thinks colleges should respond to AI. What he proposes is radical —but, if you've concluded that AI really is going to destroy everything these institutions stand for, I think you have to at least co…

>What he proposes is radical It sounds entirely reasonable and moderate to me.

Also entirely ineffective. Banning individual behavior won't prevent collective dysfunction and will only harm honest actors. The only answer that makes sense is reforming the acceptance of work to be resistant to the inevitable (ab)use of AI.

Re: The threat is comfortable drift toward not understanding what you're doing

#567

Post by jcgrillo » Sun, Apr 05, 2026, 11:50 PM UTC

Wowfunhappy wrote: post #227

> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…

It seems to me the answer is more "stay away from Bobs". If you don't get too tangled up with those guys when the shit hits the fan you won't get much on you.

Re: The threat is comfortable drift toward not understanding what you're doing

#568

Post by throwaway27448 » Sun, Apr 05, 2026, 11:51 PM UTC

dragontamer wrote: post #354

conception wrote: post #350

Earlier quoted context omitted.

Well, we are already rolling over and dying (literally) on everything from vaccine denial to climate change. So, yes, we are. Obviously yes.

In the US it is dying off. Not so in plenty of other countries. Hopefully US reverses the anti-science trend before it's too late

The replication crisis is hardly unique to the US and is greatly exacerbated with the use of AI.

Re: The threat is comfortable drift toward not understanding what you're doing

#569

Post by thepasch » Mon, Apr 06, 2026, 12:03 AM UTC

rglullis wrote: post #530

thepasch wrote: post #408

Earlier quoted context omitted.

> But to even know what is more useful, it is crucial to have walked the walk. I feel like people tend to forget that among the many things LLMs can do these days, “using a search engine” is among them. In fact, they use them better than the majority of people do! The conversation people think they’re having here and the conversation that actually needs to be had are two entirely different conversations. > I don’t kn…

> I’ve now ostensibly understood what a derivative does and what it’s used for, yet I have zero idea how to mathematically do it. Does that make any results I gain from this intuitive understanding any less valuable? From a science standpoint, I'd say whatever "results" you got are completely worthless. > I’ll generally have some sort of hypothesis of what kind of result I’m expecting, given that my understanding is…

Asking all of these questions after (allegedly) reading my entire comment either means you didn't pay attention, in which I'm not going to spend any more effort responding either; or you've completely missed the point, in which case I can probably save myself the effort anyway. In any case, if you're genuinely interested in answers to your questions instead of merely posturing, I suggest you re-read carefully and then make a better faith attempt at engaging with it.

I'll leave these direct quotes from the comment as a hint:

> But that only matters if I take its output at face value. […] If I know that a lot of unknown unknowns exist around a thing I’m working with, then I also know that unexpected results, as well as expected ones, require more thorough verification.

Re: The threat is comfortable drift toward not understanding what you're doing

#570

Post by janalsncm » Mon, Apr 06, 2026, 12:03 AM UTC

> There's a common rebuttal to this, and I hear it constantly. "Just wait," people say. "In a few months, in a year, the models will be better. They won't hallucinate. They won't fake plots. The problems you're describing are temporary."

To some extent, the reason models will get better is because companies will hire PhDs to train them on increasingly complex problems.

The problem is that more complex problems take longer to train, more time to test, require more compute, and are harder to verify. This is why “just make it bigger” is a losing proposition imo.

A lot of what I just said is also true for RLVR.