Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The threat is comfortable drift toward not understanding what you're doing

ergosphere.blog

Post reply on HN
Print view View original

111–120 of 668 posts

Re: The threat is comfortable drift toward not understanding what you're doing

#111

Post by LeonardoTolstoy »

theteapot wrote: post #93

I have a vaguely unrelated question re: > You do what your supervisor did for you, years ago: you give each of them a well-defined project. Something you know is solvable, because other people have solved adjacent versions of it. Something that would take you, personally, about a month or two. You expect it to take each student about a year ... Is that how PhD projects are supposed to work? The supervisor is a subjec…

It is a spectrum. My advisor was very hands off. He didn't, ultimately, even really understand my PhD. He knew the problem, but he had no path in mind to solve it, that was up to me. I'm now working (as a software engineer) with a person who is very hands on with his students (and even postdocs) to the point of giving them specific tasks to do and then discussing the result every week. He defines the problems and structure of the solution, the students at least partially are an extension of himself, they are doing stuff he merely doesn't have time to do himself.

And there is everything in between.

Re: The threat is comfortable drift toward not understanding what you're doing

#112

Post by thepasch »

bakugo wrote: post #84
sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

Bob can't do things, Bob's AI can do things that Bob asks it to do. And the AI can only do things that have been done before, and only up to a certain level of complexity. Once that level is reached, the AI can't do things anymore, and Bob certainly isn't going to do anything about that, because Bob doesn't know how to do anything himself. One has to question what value Bob himself even brings to the table. But let's…

The question is whether it’s more important to be able to do things, or more important to have a good sense and a keen eye for what to do at any given moment. I personally think both are really important, and I also think AI won’t be able to do both better than any human could for another while, and moreso when it comes to doing both at the same time (though I’m not going to claim it’s never going to).

My point is that both Alice and Bob have a place in this world. In fact, Bob isn’t really doing much different from what a Pricipal Investigator is already doing today in a research context.

Re: The threat is comfortable drift toward not understanding what you're doing

#113

Post by xandrius »

DavidPiper wrote: post #89

I've just started a new role as a senior SWE after 5 months off. I've been using Claude a bit in my time off; it works really well. But now that I've started using it professionally, I keep running into a specific problem: I have nothing to hold onto in my own mind. How this plays out: I use Claude to write some moderately complex code and raise a PR. Someone asks me to change something. I look at the review and thin…

Then you're using it more towards vibe coding than AI-assisted coding: I use AI to write the stuff the way I want it to be written. I give it information about how to structure files, coding style and the logic flow.

Then I spend time to read each file change and give feedback on things I'd do differently. Vastly saves me time and it's very close or even better than what I would have written.

If the result is something you can't explain than slow down and follow the steps it takes as they are taken.

Re: The threat is comfortable drift toward not understanding what you're doing

#114

Post by skydhash »

simianwords wrote: post #54

> Frank Herbert (yeah, I know I'm a nerd), in God Emperor of Dune, has a character observe: "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking; there's the real danger." Herbert was writing science fiction. I'm writing about my office. The distance between those two things has gotten uncomfortably small. The author is a bit naive here: 1. Soc…

Current civilization is very complex. And it’s also fragile in some parts. When you build systems around instant communication and the availability of stuff built in the other side of the world on a fixed schedule, it’s very easy to disrupt.

> 4. People will eventually get the hang of using AI to do the optimum amount of delegation such that they still retain what is necessary and delegate what is not necessary. People who don't do this optimally will get outcompeted

Then they’ll be at the mercy of the online service availability and the company themselves. Also there’s the non deterministic result. I can delegate my understanding of some problems to a library, a software, a framework, because their operation are deterministic. Not so with LLMs.

Re: The threat is comfortable drift toward not understanding what you're doing

#115

Post by CuriouslyC »

djaro wrote: post #14
sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> So if Bob can do things with agents, he can do things. The problem arrises when Bob encounters a problem too complex or unique for agents to solve. To me, it seems a bit like the difference between learning how to cook versus buying microwave dinners. Sure, a good microwave dinner can taste really good, and it will be a lot better than what a beginning cook will make. But imagine aspiring cooks just buying premade…

Just because Bob doesn't know e.g. Rust syntax and library modules well, doesn't mean that Bob can't learn an algorithm to solve a difficult problem. The AI might suggest classes of algorithms that could be applicable given the real world constraints, and help Bob set up an experimental plan to test different algorithms for efficacy in the situation, but Bob's intuition is still in the drivers's seat.

Of course, that assumes a Bob with drive and agency. He could just as easily tell the AI to fix it without trying to stay in the loop.

Re: The threat is comfortable drift toward not understanding what you're doing

#116

Post by samusiam »

qsera wrote: post #55
sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

>The thing is, agents aren’t going away... Aren't they currently propped up by investor money? What happens when the investors realize the scam that it is and stop investing or start investing less...

> Aren't they currently propped up by investor money?

Are Chinese model shops propped up by investor money? Is Google?

Open weights models are only 6 months behind SOTA. If new model development suddenly stopped, and today's SOTA models suddenly disappeared, we would still have access to capable agents.

Re: The threat is comfortable drift toward not understanding what you're doing

#117

Post by kelnos »

sd9 wrote: post #46
troupo wrote: post #34

Earlier quoted context omitted.

> The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. Can he? If he outsources all his thinking and understanding to agents, can he then fix things he doesn't know how to fix without agents? Any skill is practice first and foremost. If Bob has had no practice, what then?

My point is it doesn’t matter whether he can fix things without agents. The real world isn’t an exam hall where your boss tells you “no naughty AI!”, you just get stuff done, and if Bob can do that with agents, nobody cares how he did it.

But can Bob actually do that with agents, without limit? Right now, he's going to hit a ceiling at some point, and the Alices of the world will run circles around him.

The question is: will agents improve to the point that even the most capable Alices will never be needed to solve problems? Maybe? Maybe not? I'm worried that they won't improve to that degree.

And even if they do, what is the purpose of humans in this world?

Re: The threat is comfortable drift toward not understanding what you're doing

#118

Post by wglb »

efields wrote: post #26

I literally don't know how compilers work. I've written code for apps that are still in production 10 years later.

No problem with that.

However, at one point in my career, I was frustrated with limitations in a language (Fortran II) and my curiosity got the better of me and I studied compilers thoroughly.

This led to a new job and the understanding of many new useful programming concepts. Very rewarding.

But if you are curious, studying compilers, maybe even writing a new one, will give you tools to do other things.

While working with LLMs, much of my experience gives me new ideas to push the LLM to explore.

Re: The threat is comfortable drift toward not understanding what you're doing

#119

Post by loveparade »

QuantumNomad_ wrote: post #57
sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> if Bob can do things with agents, he can do things I’ve been reminded lately of a conversation I had with a guy at hacker space cafe around ten years ago in Berlin. He had been working as a programmer for a significantly longer time than me. Long enough that for many years of his career, he had been programming in assembly. He was lamenting that these days, software was written in higher level languages, and that m…

I see these analogies a lot, but I don't like them. Assembly has a clear contract. You don't need to know how it works because it works the same way each time. You don't get different outputs when you compile the same C code twice.

LLMs are nothing like that. They are probabilistic systems at their very core. Sometimes you get garbage. Sometimes you win. Change a single character and you may get a completely different response. You can't easily build abstractions when the underlying system has so much randomness because you need to verify the output. And you can't verify the output if you have no idea what you are doing or what the output should look like.

Re: The threat is comfortable drift toward not understanding what you're doing

#120

Post by thijson »

jacquesm wrote: post #25
ghc wrote: post #20

As straw men go, this is an attractive one, but... When I was fresh out of undergrad, joining a new lab, I followed a similar arc. I made mistakes, I took the wrong lessons from grad student code that came before mine, I used the wrong plotting libraries, I hijacked python's module import logic to embed a new language in its bytecode. These were all avoidable mistakes and I didn't learn anything except that I should…

AI does not give you knowledge. It magnifies both intelligence and stupidity with zero bias towards either. If you were above average intelligent then you may be able to do a little bit more than before assuming you were trained before AI came along. And if you were not so smart then you will be able to make larger messes. The problem, and I think the article indirectly points at that, is that the next generation to…

Maybe the solution is for an AI that acts as an instructor instead of just trying to solve everything itself. I do this with my kids, they ask me how to do something. I will give them hints, but not outright do it all for them. The article writer in the first part mentioned that this is how they would instruct too.
Post reply on HN