Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The threat is comfortable drift toward not understanding what you're doing

ergosphere.blog

Post reply on HN
Print view View original

571–580 of 668 posts

Re: The threat is comfortable drift toward not understanding what you're doing

#571

Post by somesortofthing »

dummydummy1234 wrote: post #557
somesortofthing wrote: post #545

I used to feel this way but... honestly, I've found that pressing on with only a vague understanding of what's happening and then diving deep with the agent's own help if it keeps making bad decisions leads to more output of comparable quality. Even without a deep understanding of the topic, you can usually tell when the LLM is BSing and you need to intervene. The model has much more knowledge "present-at-hand" than…

I think there is a difference between things like coding where it is semi closed loop, at the end of the day the software works or not. Vs fields where there is not a reliable feedback path, or that feedback path is much more noisy.

There definitely is but even then, you can get a feel for a loop for more open-ended tasks too - you move forward until the model output starts to look handwavy/contradictory, then pause to talk to it/consult outside sources to improve your own knowledge. Most "fuzzy" fields also have quantitative components, and it's often worth stopping for a moment to put together some kind of quantitative evaluation suie to give the model grounding. When you've learned the right path yourself, you start moving forward again. It's for sure slower and more error-prone if you were already an expert when you started, but it's workable, and head-and-shoulders better than what you could do without the AI.

Re: The threat is comfortable drift toward not understanding what you're doing

#572

Post by spacechild1 »

xnzakg wrote: post #524
spacechild1 wrote: post #276

Earlier quoted context omitted.

That's one of several reasons why I'm trying not to rely too much on LLMs. The prospect of only being able to code with a working internet connection and a subscription to some megacorp service is not particularly appealing to me.

Local/open LLMs are a thing though. You can build a server for hosting decent sized (100-200B) models at home for a few k$. They may not be Opus-level, but hopefully we can get something matching current SOTA, but that we can run locally, before the megacorps get too greedy. Alternatively you could find some other people to share the HW cost and run some larger models (like Kimi-K2.5 at 1.1T params).

> You can build a server for hosting decent sized (100-200B) models at home for a few k$.

That's definitely not an option for me :-D

True open LLMs could be a viable solution in the future, but only if they can be operated and sustained on a community basis. I have too little insights into the actual costs of running such models to judge whether this would be feasible. Then there is always the problem of how to deal with bad actors. This is all but trivial.

At the moment, I'd rather spend time working on sharpening my actual programming and thinking skills :) I actually enjoy the act of programming and see it as part of my creative expression. Fortunately, I don't code for a living (at least not directly), so nobody can tell me how to write my software.

Re: The threat is comfortable drift toward not understanding what you're doing

#573

Post by enraged_camel »

greenchair wrote: post #232
xandrius wrote: post #113

Earlier quoted context omitted.

Then you're using it more towards vibe coding than AI-assisted coding: I use AI to write the stuff the way I want it to be written. I give it information about how to structure files, coding style and the logic flow. Then I spend time to read each file change and give feedback on things I'd do differently. Vastly saves me time and it's very close or even better than what I would have written. If the result is somethi…

AI assisted coding makes you dumber full stop. It's obvious as soon as you try it for the first time. Need a regex? No need to engage your brain. AI will do that for you. Is what it produced correct? Well who knows? I didn't actually think about it. As current gen seniors brains atrophy over the next few years the scarier thing is that juniors won't even be learning the fundamentals because it is too easy to let AI h…

>> AI assisted coding makes you dumber full stop. It's obvious as soon as you try it for the first time. Need a regex? No need to engage your brain. AI will do that for you.

Regex is the worst possible example you could have given. Seriously, how many people do you know who painstakingly hand-craft their own regexes as opposed to using one of the million tools out there that can work backwards from example inputs and outputs to generate a regex that satisfies the conditions?

Re: The threat is comfortable drift toward not understanding what you're doing

#574

Post by pessimizer »

sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things.

But Bob can't do things with agents.

He can get a project from someone else and ask the agents to do that project. Then give the output of the agents back to that someone else, and that someone else reviews it, says why it's wrong, and sends it back. Bob feeds the review to the agents, gets something back, then gives the output back to that someone else who reviews it, etc. So,

1) The loop requires his advisor to know how to go about doing the thing.

2) Bob is absolutely unnecessary and should be discarded.

3) Alice will eventually be qualified to be an advisor.

edit: And the crisis that the article is really pointing out is that when the advisor is using the LLM (while Bob is driving an Uber), and his productivity goes way up because he's only handling the things only he can handle, what about Alice?

Let's say that pre-AI the advisor could either do the job in 2 months or assign it to Alice who could do it in 12 with a week of the advisor's supervision/review. Now, with the LLM, the advisor can do the job in 2 weeks without Alice. Before, Alice made barely any money and had no health insurance. After, Alice is also driving Uber.

Now the advisor has a heart attack and now the thing just can't be done. Also, Ubers become pretty much self-driving, so Bob and Alice are not only ignorant, but unemployed. They can't even afford to take an Uber.

Re: The threat is comfortable drift toward not understanding what you're doing

#575

Post by Jensson »

thepasch wrote: post #374
Jensson wrote: post #309

Earlier quoted context omitted.

> I don’t agree that doing everything yourself manually is the only thing that can grant you that understanding, because I don’t think that understanding is domain-specific. It evolves naturally as soon as someone realizes that their list of unknown unknowns is FAR larger than their list of known anythings, and that the first step in attempting to solve a problem is to prune that list as far as you can get it while r…

> This isn’t true, a car mechanic never evolves into an engineer, a nurse never evolve into a doctor. “Doesn’t generally happen” =/= “is literally impossible”. The word “never” should be used with care. > A car mechanic can learn to do some tasks you normally need an engineer for and same with nurses This statement can only make sense if you regard titles as something that’s imbued upon you, and until it is, you are…

You didn't read my closing statement:

> There are maybe some exceptions to this, but those exceptions are so rare that it doesn't matter for this discussion. A few people still learning it properly wont save anything.

It is fine it was stupid of me to not go and fix the "never" earlier but instead corrected it at the end of my post. You are right it would be very dumb if you thought people never ever could make such a transition, but I wrote that as the weaker version of never. There is a reason there is a "never ever", since often in common language "never" doesn't mean never.

Re: The threat is comfortable drift toward not understanding what you're doing

#576

Post by danorama »

Izkata wrote: post #532
vinceguidry wrote: post #444

Earlier quoted context omitted.

I was there for that war. Kids weren't listening and didn't care back then either. If anything, Gen Z is the most politically-aware generation we've had since we started keeping track. Trump doesn't have to justify a single thing because the billionaires behind him know that every last bet is off and their very livelihoods are at risk, and his entire base of support up and down the chain are either complicit or foole…

Trump was in part a response to that realization, an outsider from both parties that used the Republicans to get in the door. Bernie Sanders was supposed to be the same, and when he dropped out a lot of us switched to Trump because they represented the same thing. Tearing down entrenched establishments that don't work for the people is what people were voting for in 2016 and were disappointed he didn't do it. They're…

"Bernie Sanders was supposed to be the same, and when he dropped out a lot of us switched to Trump because they represented the same thing."

That is definitely an opinion.

Re: The threat is comfortable drift toward not understanding what you're doing

#577

Post by Jensson »

thepasch wrote: post #416
Planktonne wrote: post #387

Earlier quoted context omitted.

> the idea that this category of people will never be able to improve into the first category of people The fundamental difference between the categories is that the first is filled with people who put the effort in to learning/understanding, and the second is filled with people who take the shortcut around learning/understanding. Changing from the second category to the first is something that would require already…

> The fundamental difference between the categories is that the first is filled with people who put the effort in to learning/understanding, and the second is filled with people who take the shortcut around learning/understanding. Exactly! That’s my entire point. Because now you’re separating the categories by “is willing to put in effort” and “is not willing to put in effort” rather than by “has done the thing” and…

> I think the disagreement doesn’t lie in this concept, but rather in whether an LLM can be used by someone who’s willing to put in effort to assist them in doing so, rather than just having it do it for them

No, you misunderstood here. People aren't saying "it is harder to learn in the future", the issue is "it will be harder to make sure that someone will learn in the future".

Currently you need an engineering degree and experience to do engineering work. However if in the future a lot of people get their degree and experience just by calling LLM for every problem, those engineers will not understand at all what they are doing. Before someone having that experience will have had solved a lot of problems manually on the job, that experience made them an expert. The same person solving those by calling an LLM and pasting in the answer will just as ignorant as someone with no experience.

Most such people today didn't wanna learn to be engineers out of curiosity, they just wanted a job. In the future all such people would use LLM and never learn. Those are the main parts of our workforce, so it is a scary prospect that in the future we cannot force them to learn things properly in the same way since LLM allows them to do basic tasks without learning.

If you argue there are plenty of people who learn for fun, then you would be wrong. Extremely few people learn enough in their own time to contribute meaningfully to for example mathematics, it isn't enough to matter. People learn those fundamentals primarily because they are forced to do it for a degree they need for a job, if they weren't forced to learn and pass tests they would happily go do the job without any knowledge or skills.

Re: The threat is comfortable drift toward not understanding what you're doing

#579

Post by irishcoffee »

Wowfunhappy wrote: post #522
pxc wrote: post #453

Earlier quoted context omitted.

I was pretty interested until I got to this part: > Another reason that a no-exceptions policy is important: If students with disabilities are permitted to use laptops and AI, a significant percentage of other students will most likely find a way to get the same allowances, rendering the ban useless. I witnessed this time and again when I was a professor—students without disabilities finding ways to use disability ac…

You don't have to agree with his precise solution, and in fact I'm not sure whether I do. However, I found the article useful because it got me thinking about the universe of things we could be considering, if we really do think AI is poised to destroy education as we know it.

I don’t know about anyone else here, but college was not educating because I was at college. I did all of the reading and studying on my own. The classes weren’t very interesting, most of my TAs didn’t speak the native language well at all, nor did half the professors.

I enjoyed my time, I made a lot of lifelong friends, and figured out how to live on my own. My buddies that enrolled in boot camp instead of college learned all those same skills, for free.

Education won’t be ruined or blemished my LLMs, the whole thing was a joke to begin with. The bit that ruined college was unlimited student loans… and all of our best and brightest folks running the colleges raping students for money. It’s pathetic, evil, and somehow espoused.

Re: The threat is comfortable drift toward not understanding what you're doing

#580

Post by Ancalagon »

techblueberry wrote: post #337
caxap wrote: post #292

If this article was written a year ago, I would have agreed. But knowing what I know today, I highly doubt that the outcomes of LLM/non-LLM users will be anywhere close to similar. LLMs are exceptionally good at building prototypes. If the professor needs a month, Bob will be done with the basic prototype of that paper by lunch on the same day, and try out dozens of hypotheses by the end of the day. He will not be ch…

The problem is in most environments Bob won’t spend the rest of the year figuring out what the LLM did, because bob will be busy promoting the LLM for the next deliverable, and the problem is that if all bob has time for us to prompt LLMs, and not understand, there will be a ceiling to Bob’s potential. This won’t affect everyone equally. Some Bob’s will nerd out and spend their free time learning, but other Bob’s won…

Prompt not promote*
Post reply on HN