Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI ruining our skills? Early results are in – and they're not good

nature.com

Post reply on HN
Print view View original

281–290 of 328 posts

Re: Is AI ruining our skills? Early results are in – and they're not good

#281

Post by CuriouslyC »

iLoveOncall wrote: post #258
CuriouslyC wrote: post #29

Earlier quoted context omitted.

I recently did a manual exercise to force myself to keep my skills from decaying too much after about a year of using agents exclusively. My ability to go from a blank slate to software was indeed in the toilet, but my ability to reason over and edit code seems to be surviving fine. I suspect that your LLM-pilled coworkers' judgment issues are related to laziness that LLMs have enabled, rather than an inherent proper…

> My ability to go from a blank slate to software > my ability to reason over and edit code You can't be able to do one but not the other, they're the same thing. > I suspect that your LLM-pilled coworkers' judgment issues are related to laziness that LLMs have enabled, rather than an inherent property of LLM use. Does it matter? The end result is the same. Maybe the studies that the article mentions are simply showc…

Not true, blank page syndrome, inertia and write vs edit flows are a known thing across disciplines. You might not think about it as much because you have a command in your package manager or framework to bootstrap projects, but imagine if you had to hand code _everything_. I'll guess there are languages/frameworks where you consider yourself quite proficient but where you couldn't bootstrap a project without checking the docs if not for the tooling.

It matters. If LLMs are like alcohol, useful but potentially dangerous, they should be treated very differently than if they're just poison. If someone's a drunk and they can't keep their lives together, it's not the alcohol's fault, it's the person's.

Re: Is AI ruining our skills? Early results are in – and they're not good

#282

Post by wvenable »

fzeroracer wrote: post #280
wvenable wrote: post #261

Earlier quoted context omitted.

How can I argue about such a hypothetical?!? If I ask an LLM or my mother-in-law about how to get wine out of the carpet, and I get wine out of the carpet, maybe I have done it wrong and my house will burn down! Who knows?! Your argument is meaningless; there's nothing anyone can say to that.

And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it? When you ask 'who knows' that's the point of research which was my original comment here. The same goes true for some random asshole telling you to drink bleach as it does an LLM, except people seemingly have hyped themselves into believing the LLM is more right somehow instead of being trained on ev…

> And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it?

But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. It's not perfect and I don't expect it to be perfect. According to my personal experience it's definitely good enough for a lot of things like recipes, coding, hobby stuff, and home repairs.

You're expecting me not to trust my own personal learned experience using AI as if somehow all my continued successes are worth nothing because you can invent some wild hypothetical. The more I use it, the more I understand its limitations and avoid them.

Re: Is AI ruining our skills? Early results are in – and they're not good

#283

Post by ianbutler »

skjfjnflw wrote: post #273
ianbutler wrote: post #244

Earlier quoted context omitted.

> Turns out for most people, there's not much better to do during your 9 to 5 than solving problems for half a million a year. Lol. I don't know I was talking with a guy yesterday who left a FAANG clearing 1.4 million / yr who's now running his own successful startup. My sample is successful founders backed by top tier VCs or exited founders who have done much better than FAANG. If you can't do more you stick around…

You're talking about what happens to engineers who decide they want to be small business owners rather than engineers. That's orthogonal to this thread about where engineers with the best technical skills end up.

Most of the best engineers I know leave faang because the work at faang sucks. It is not orthogonal. The majority of faang work is not highly impactful work like trivially because the amount of work outstrips the amount of interesting work by an easy oom.

FAANG doesn't straightforwardly retain the best technical talent, and the median task there is routine.

Re: Is AI ruining our skills? Early results are in – and they're not good

#284

Post by themafia »

orbital-decay wrote: post #262
themafia wrote: post #236

Earlier quoted context omitted.

> Human attention is truly ephemeral, with a ridiculously short span. I do not believe this at all. I think you'd have to have a very limited experience working with other human beings to be able to believe this. > and by training it better. "Oh yea, just do it _better_. That's your problem." Perhaps some people operate without any context but most of us find the experience lacking.

That's a simple neurologic fact, there's nothing to believe. You can use a trivial experiment to verify that you can't keep details in your sliding attention window for more than a few seconds or focus on more than a few things simultaneously. Human memory and cognition is layered and this immediate layer is what resembles the model's context the most. You're possibly mixing it up with long-term memory which doesn't…

> You can use a trivial experiment to verify that you can't keep details in your sliding attention window for more than a few seconds or focus on more than a few things simultaneously.

I said "context window" not "attention window." Of course spans of attention are limited. Knowledge is not. Knowledge is often highly specific.

> You're possibly mixing it up

Not really. You've simply failed to verify your understanding of my argument and instead created something of a strawman.

> You aren't keeping the entire codebase in your memory, just its highly processed and conceptualized version

And what is your basis for this claim? Why a codebase? You don't think I can't remember an entire function? Yet actors can remember entire sets of lines for a scene? Is that just a highly processed and abstracted version in their minds? And they just run some cognitive loop to recreate dialog in real time?

> they need stronger processing of what they already remember.

What is "stronger?" More time? More memory? More compute? And how is that put to use? Why is it, when given certain prompts, that LLMs reproduce 100s of pages of directly copied and copyrighted work?

> Models can be trained better to cram more intelligence into the same amount of parameters, that's what I mean.

Cool. _How_? What is the limit of this training? How efficient is it? How many resources do you need on the input for a given increase in output? Otherwise it's just a ton of hand waving going on here.

Re: Is AI ruining our skills? Early results are in – and they're not good

#285

Post by fzeroracer »

wvenable wrote: post #282
fzeroracer wrote: post #280

Earlier quoted context omitted.

And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it? When you ask 'who knows' that's the point of research which was my original comment here. The same goes true for some random asshole telling you to drink bleach as it does an LLM, except people seemingly have hyped themselves into believing the LLM is more right somehow instead of being trained on ev…

> And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it? But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. It's not perfect and I don't expect it to be perfect. According to my personal experience it's definitely good enough for a lot of things like recipes, coding, hobby stuff, and home repairs…

> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can.

No, no, no and no. This is the biggest mistake I see people consistently make with LLMs. It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt.

If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink and 1 doctor who disagrees, it will by virtue of probability tell you to drink bleach. Companies add additional safeguards or system prompts to try and keep it 'on rails' but it's all probability based on what you prompt it. It is by the literal functionality of how it works to do so. A function depending on what said company ingests during training, many of which include the entire corpus of the internet.

If you do not understand how LLMs work under the hood then yes, you shouldn't trust your own personal learned experience because you've already demonstrated that you're wrong.

Re: Is AI ruining our skills? Early results are in – and they're not good

#286

Post by wvenable »

fzeroracer wrote: post #285
wvenable wrote: post #282

Earlier quoted context omitted.

> And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it? But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. It's not perfect and I don't expect it to be perfect. According to my personal experience it's definitely good enough for a lot of things like recipes, coding, hobby stuff, and home repairs…

> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. No, no, no and no. This is the biggest mistake I see people consistently make with LLMs. It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt. If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink and 1…

> It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt.

That's not specifically true either; training is more complex than that. ChatGPT had to be trained, for example, to answer questions in a chat format.

> If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink..

Again with the hypotheticals! You literally cannot discuss this subject without hallucinating things that don't exist. LLMs are trained on huge corpus of information from books to videos to reddit posts. Ultimately, statistically, it's going to predict the most common answer to something. Yes, that might be wrong but the vast majority of the time it's going to be right. And you know what, in the real non-hypothetical world, it works great. As much you don't want it to. You can hypothetically hallucinate as many weird unlikely scenarios as you want but that doesn't make it true.

Re: Is AI ruining our skills? Early results are in – and they're not good

#287

Post by fzeroracer »

wvenable wrote: post #286
fzeroracer wrote: post #285

Earlier quoted context omitted.

> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. No, no, no and no. This is the biggest mistake I see people consistently make with LLMs. It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt. If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink and 1…

> It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt. That's not specifically true either; training is more complex than that. ChatGPT had to be trained, for example, to answer questions in a chat format. > If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink.. Again with the hypotheticals! You literally cannot…

I see there is no point trying to get through to you. These are not hypotheticals, as an example just recently ChatGPT advised a user to mix bleach and vinegar. I have seen Googles AI hallucinate many details that I had to double check and would've been disastrous had I assumed it was truthy. There's a reason why models have the legal clauses like 'this is for entertainment purposes only' because they are trying to shield themselves from legal obligations when it makes catastrophic mistakes.

The people least willing to understand how the system works are also the most willing to blindly believe it.

Re: Is AI ruining our skills? Early results are in – and they're not good

#288

Post by somebehemoth »

wvenable wrote: post #282
fzeroracer wrote: post #280

Earlier quoted context omitted.

And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it? When you ask 'who knows' that's the point of research which was my original comment here. The same goes true for some random asshole telling you to drink bleach as it does an LLM, except people seemingly have hyped themselves into believing the LLM is more right somehow instead of being trained on ev…

> And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it? But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. It's not perfect and I don't expect it to be perfect. According to my personal experience it's definitely good enough for a lot of things like recipes, coding, hobby stuff, and home repairs…

> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can.

This is not true. This is not how LLM's work. They have no concept of accuracy or "best".

Re: Is AI ruining our skills? Early results are in – and they're not good

#289

Post by wvenable »

fzeroracer wrote: post #287
wvenable wrote: post #286

Earlier quoted context omitted.

> It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt. That's not specifically true either; training is more complex than that. ChatGPT had to be trained, for example, to answer questions in a chat format. > If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink.. Again with the hypotheticals! You literally cannot…

I see there is no point trying to get through to you. These are not hypotheticals, as an example just recently ChatGPT advised a user to mix bleach and vinegar. I have seen Googles AI hallucinate many details that I had to double check and would've been disastrous had I assumed it was truthy. There's a reason why models have the legal clauses like 'this is for entertainment purposes only' because they are trying to s…

I understand how the system works. I also know that the free LLMs are dumb as rocks. But they're not all dumb as rocks -- it's a continuum. You're the person who doesn't seem to understand that it's not all or nothing.

AI companies are trying to build a system that gives the best and most accurate answers possible -- that's the whole point of it all.

Re: Is AI ruining our skills? Early results are in – and they're not good

#290

Post by wvenable »

somebehemoth wrote: post #288
wvenable wrote: post #282

Earlier quoted context omitted.

> And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it? But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. It's not perfect and I don't expect it to be perfect. According to my personal experience it's definitely good enough for a lot of things like recipes, coding, hobby stuff, and home repairs…

> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. This is not true. This is not how LLM's work. They have no concept of accuracy or "best".

You missed my point.

The system designers (OpenAI, Anthropic, etc.) are absolutely trying to build a system that gives the best and most accurate answers possible.

The model itself does not have goals, intentions, or an internal concept of "best" or "accurate."

Post reply on HN