Live data from Hacker News

The threat is comfortable drift toward not understanding what you're doing

ergosphere.blog

371–380 of 668 posts

Re: The threat is comfortable drift toward not understanding what you're doing

#371
I dunno. Claude helped me implement a new memory allocator, compacting garbage collector and object heap for my programming language. I certainly understood what I was doing when I did this. The experience was extremely engaging for me. Claude taught me a lot.

I think the real danger is no longer caring about what you're doing. Yesterday I just pointed Claude at my static site generator and told it to clean it up. I wanted to care but... I didn't.

Re: The threat is comfortable drift toward not understanding what you're doing

#372

Nobody actually understands what they're doing. When you're learning electronics, you first learn about the "lumped element model" which allows you to simplify Maxwell's equations. I think it is a mistake to think that solving problems with a programming language is "knowing how to do things" - at this point, we've already abstracted assembly language -> machine instructions -> logic gates and buses -> transistors an…

> as if debugging python code is the point of it all.

You have a good point, but I would argue that debugging itself is a foundational skill. Like imagine Sherlock Holmes being able to use any modern crime-fighting technology, and using it extensively. If Sherlock is not using his deductive reasoning, then he's not a 'detective'. He's just some schmuck who has a cool device to find the right/wrong person to arrest.

Debugging is "problem-solving" in a specific domain. Sure, if the problem is solved, then I guess that's the point of it all and you don't have to solve the problem. But we're all looking towards a world in which people have to solve problems, but their only problem-solving skill is trying to get an AI to find someone to arrest. We need more Sherlocks to use their minds to get to the bottom of things, not more idiot cops who arrest the wrong person because the AI told them to.

Re: The threat is comfortable drift toward not understanding what you're doing

#373

> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…

> And so the paradox is, the LLMs are only useful† if you're Schwartz, and you can't become Schwartz by using LLMs. I have gained a lot of benefit using LLMs in conjunction with textbooks for studying. So, I think LLMs could help you become Schwartz.

How do you know you have?

Re: The threat is comfortable drift toward not understanding what you're doing

#374

Earlier quoted context omitted.

> “People who prompt things” can only do the latter, and they routinely do it poorly. Right, but what I don’t agree with here is the idea that this category of people will never be able to improve into the first category of people. The value of an experienced anything is that they realize there is a big chasm between something that works now and something that will continue to work long into the future. I don’t agree…

> I don’t agree that doing everything yourself manually is the only thing that can grant you that understanding, because I don’t think that understanding is domain-specific. It evolves naturally as soon as someone realizes that their list of unknown unknowns is FAR larger than their list of known anythings, and that the first step in attempting to solve a problem is to prune that list as far as you can get it while r…

> This isn’t true, a car mechanic never evolves into an engineer, a nurse never evolve into a doctor.

“Doesn’t generally happen” =/= “is literally impossible”. The word “never” should be used with care.

> A car mechanic can learn to do some tasks you normally need an engineer for and same with nurses

This statement can only make sense if you regard titles as something that’s imbued upon you, and until it is, you are incapable of performing the acts that someone who has earned that tile can perform. I’ll just say I fundamentally disagree with this notion on pretty much every conceivable level, and if that’s the belief system you subscribe to, that would also makes arguing about this any further pointless. But I might just be getting you wrong.

Re: The threat is comfortable drift toward not understanding what you're doing

#375
This "drift" is not a drift at all, nor is it new. There are many names for this such as cargo cult and think-by-numbers (like paint by numbers), ant mills. It is recipes. And many, many common recipes demonstrate a wide spread lack of understanding.

This kind of follow-the-leader kind of "thinking" is probably a requirement. The amount of expertise it would require to understand and decide about things in our daily life is overwhelming. Do you fix your own car, decide each day how to travel, get food and understand how all that works? No.

So what is the problem? The problem is that if you follow the leader and the leader has an agenda that differs from your agenda. Do you really think that with Jeff Bezos being a (the?) major investor in Washington Post has anything to do with Democraccy? You know as in the WAPO slogan "Democracy dies in the Dark".

Does Jeff have an agenda that differs from yours? Yes. NYT? Yes. Hacker news? Yes. Google? Yes. We now live in a world so filled with propaganda that it makes no difference whether something is AI. We all "follow". Or not.

Re: The threat is comfortable drift toward not understanding what you're doing

#377

> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…

[deleted]

Re: The threat is comfortable drift toward not understanding what you're doing

#378
post #336

Earlier quoted context omitted.

> There is no history of any sort of long planning Sure there is. Its the formal education system that produced the college grad.

… between employees and employers. The proposal that everyone pay for college until they are in their 40s doesn’t seem viable.

Maybe, but there is a trend towards more and longer education. More college graduates, more PhD grads, etc.

Re: The threat is comfortable drift toward not understanding what you're doing

#379
post #162

Earlier quoted context omitted.

People would have said the same about graphing calculators or calculators before that. Socrates said the same thing about the written word. The determining factor is always "did I come up with this tool". Somehow, subsequent generations always manage to find their own competencies (which, to be fair, may be different). This isn't guaranteed to play out, but it should be the default expectation until we actually see g…

> People would have said the same about graphing calculators or calculators before that. As it happens, we generally don't let people use calculators while learning arithmetic. We make children spend years using pencil and paper to do what a calculator could in seconds.

This is why I don’t understand the calculator analogy. Letting beginners use LLMs is like if we gave kids calculators in 1st grade and told Timmy he never needs to learn 2 + 2. That’s not how education works today.

Re: The threat is comfortable drift toward not understanding what you're doing

#380
post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

[dead]
Post reply on HN