Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The threat is comfortable drift toward not understanding what you're doing

ergosphere.blog

Post reply on HN
Print view View original

61–70 of 668 posts

Re: The threat is comfortable drift toward not understanding what you're doing

#61

Post by torben-friis »

sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

Can you run an industry level LLM at home?

If not, you're changing learning to cook for Uber only meals.

And since the alternative is starving, Uber will boil the pot.

Don't give up your self sufficiency.

Re: The threat is comfortable drift toward not understanding what you're doing

#62

Post by mattmanser »

roenxi wrote: post #40
djaro wrote: post #14

Earlier quoted context omitted.

> So if Bob can do things with agents, he can do things. The problem arrises when Bob encounters a problem too complex or unique for agents to solve. To me, it seems a bit like the difference between learning how to cook versus buying microwave dinners. Sure, a good microwave dinner can taste really good, and it will be a lot better than what a beginning cook will make. But imagine aspiring cooks just buying premade…

That doesn't sound like much of an issue. Bob was already going to encounter problems that are too large and complex for him to solve, agents or otherwise. Life throws us hard problems. I don't recall if we even assumed Bob was unusually capable, he might be one of life's flounderers. I'd give good odds that if he got through a program with the help of agents he'll get through life achieving at least a normal level o…

The authors point went a little over your head.

It doesn't matter if Bob can be normal. There was no point to him being paid to be on the program.

From the article:

If you hand that process to a machine, you haven't accelerated science. You've removed the only part of it that anyone actually needed.

Re: The threat is comfortable drift toward not understanding what you're doing

#63

Post by selimthegrim »

stavros wrote: post #3

I see this fallacy being committed a lot these days. "Because LLMs, you will no longer need a skill you don't need any more, but which you used to need, and handwaves that's bad". Academia doesn't want to produce astrophysics (or any field) scientists just so the people who became scientists can feel warm and fuzzy inside when looking at the stars, it wants to produce scientists who can produce useful results. Bob pr…

Completely missed the point of the blog post which was that the point was producing the scientist not the result

Re: The threat is comfortable drift toward not understanding what you're doing

#64

Post by sd9 »

QuantumNomad_ wrote: post #57
sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> if Bob can do things with agents, he can do things I’ve been reminded lately of a conversation I had with a guy at hacker space cafe around ten years ago in Berlin. He had been working as a programmer for a significantly longer time than me. Long enough that for many years of his career, he had been programming in assembly. He was lamenting that these days, software was written in higher level languages, and that m…

Lovely story, thanks for sharing.

I wonder how many assembly programmers got over it and retrained, versus moved on to do something totally different.

I find the agentic way of working simultaneously more exhausting and less stimulating. I don’t know if that’s something I’m going to get over, or whether this is the end of the line for me.

Re: The threat is comfortable drift toward not understanding what you're doing

#65

Post by sd9 »

torben-friis wrote: post #61
sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

Can you run an industry level LLM at home? If not, you're changing learning to cook for Uber only meals. And since the alternative is starving, Uber will boil the pot. Don't give up your self sufficiency.

I’m very good at the handcrafted stuff, I’ve been doing this a while. I don’t feel like giving up my self sufficiency, I just feel like the writing is on the wall.

Re: The threat is comfortable drift toward not understanding what you're doing

#66

Post by skydhash »

stavros wrote: post #12
djaro wrote: post #10

Earlier quoted context omitted.

The problem is that LLMs stop working after a certain point of complexity or specificity, which is very obvious once you try to use it in a field you have deep understanding of. At this point, your own skills should be able to carry you forward, but if you've been using an LLM to do things for you since the start, you won't have the necessary skills. Once they have to solve a novel problem that was not already solved…

Then test Bob on what you actually want him to produce, ie novel problems, instead of trivial things that won't tell you how good he is. Why is it a problem of the LLM if your test is unrelated to the performance you want?

What people forget about programming is it is a notation for formal logic, one that can be executed by a machine. That formal logic is for solving a problem in the real world.

While we have a lot of abstractions that solve some subproblems, there still need to connect those solutions to solve the main problem. And there’s a point where this combination becomes its own technical challenge. And the skill that is needed is the same one as solving simpler problems with common algorithms.

Re: The threat is comfortable drift toward not understanding what you're doing

#67

Post by throwaway132448 »

The flip side I don’t see mentioned very often is that having a product where you know how the code works becomes its own competitive advantage. Better reliability, faster fixes and iteration, deeper and broader capabilities that allow you to be disruptive while everything else is being built towards the mean, etc etc. Maybe we’ve not been in this new age for long enough for that to be reflected in people’s purchasing criteria, but I’m quite looking forward to fending off AI-built competitors with this edge.

Re: The threat is comfortable drift toward not understanding what you're doing

#68

Post by Paradigma11 »

pards wrote: post #7
stavros wrote: post #3

I see this fallacy being committed a lot these days. "Because LLMs, you will no longer need a skill you don't need any more, but which you used to need, and handwaves that's bad". Academia doesn't want to produce astrophysics (or any field) scientists just so the people who became scientists can feel warm and fuzzy inside when looking at the stars, it wants to produce scientists who can produce useful results. Bob pr…

> Take away the agent, and Bob is still a first-year student who hasn't started yet. The year happened around him but not inside him. He shipped a product, but he didn't learn a trade. We're minting an entire generation of people completely dependent on VC funding. What happens if/when the AI companies fail to find a path to profitability and the VC funding dries up?

What will happen is pretty obvious. Those companies will either be classified as too important to fail and get government support or go bankrupt and will be bought for pennies on the dollar. For the customers nothing much will change since tokens are getting cheaper every year and the business is already pretty profitable. Progress will slow down massively till local open weight models catch up to pre-crash SotA and go on from there.

Re: The threat is comfortable drift toward not understanding what you're doing

#69

Post by torben-friis »

sd9 wrote: post #65
torben-friis wrote: post #61

Earlier quoted context omitted.

Can you run an industry level LLM at home? If not, you're changing learning to cook for Uber only meals. And since the alternative is starving, Uber will boil the pot. Don't give up your self sufficiency.

I’m very good at the handcrafted stuff, I’ve been doing this a while. I don’t feel like giving up my self sufficiency, I just feel like the writing is on the wall.

By "you" I actually meant this hypothetical person who's only good enough for AI assisted. Though even for us who are already experienced, we should keep the manual stuff even if it's just as going to the gym. I don't see myself retaining my skills for long by just reviewing LLM output.

Re: The threat is comfortable drift toward not understanding what you're doing

#70

Post by thijson »

defrost wrote: post #17
stavros wrote: post #5

Earlier quoted context omitted.

Yeah, it was saying that what matters is the process of training people to be good scientists, so they can produce other, more useful, results. That's literally what training is, everywhere. This argument boils down to "don't use tools because you'll forget how to do things the hard way", which nobody would buy for any other tool, but with LLMs we seem to have forgotten that line of reasoning entirely.

> This argument boils down to "don't use tools because you'll forget how to do things the hard way", which nobody would buy for any other tool, This is false. There absolutely are people that fall back on older tools when fancy tools fail. You will find such people in the military, in emergency services, in agriculture, generally in areas where getting the job done matters. Perhaps you're unfamiliar. They other week…

In the 80's the Americans thought that the Russians were backwards to be still using vacuum tubes in their military vehicles. Later they found out that they were being used because they are more tolerant to EMP from a nuclear blast.
Post reply on HN