Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Workers with less experience gain the most from generative AI

mitsloan.mit.edu

Post reply on HN
Print view View original

11–20 of 120 posts

Re: Workers with less experience gain the most from generative AI

#11

Post by smartmic »

camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

Is it really "teaching" that is being conveyed? I mean, yes, you get a suggestion for a mostly correct solution, or it points in the right direction. But how much has your brain really been reprogrammed and "learned" in the process? I fear that our problem-solving skills are somehow short-circuited and we rely too much on LLMs to spit out the most likely solutions. Alternatives, hard thinking and exploration may be left behind. So in what discipline will the inexperienced become more experienced? Prompt engineering, copy-pasting, short-cut action … but what about critical thinking, understanding and meta-learning?

Re: Workers with less experience gain the most from generative AI

#14

Post by jonplackett »

This is exactly the opposite of my experience with it.

Getting it to code, it can save me, a fairly experienced coder, quite a lot of time because it can do some boring things - boilerplate stuff, or things I can’t be bothered to look up. And I can tell if it’s done it right or not.

I’ve seen people with less coding experience use it and just take its suggestions at face value with painful results and definite detriment to their learning.

The best test I’ve seen for what it’s good at is things that are hard to do, but easy to check.

Re: Workers with less experience gain the most from generative AI

#15

Post by falcor84 »

smartmic wrote: post #11
camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

Is it really "teaching" that is being conveyed? I mean, yes, you get a suggestion for a mostly correct solution, or it points in the right direction. But how much has your brain really been reprogrammed and "learned" in the process? I fear that our problem-solving skills are somehow short-circuited and we rely too much on LLMs to spit out the most likely solutions. Alternatives, hard thinking and exploration may be l…

I think you get as much, or even more, as when asking someone more senior. You get the benefit of "scaffolding" when working on a task at the limit of your ability, and it's probably even better than asking a human, since you're forced to stay on your toes, in case it's hallucinating.

Re: Workers with less experience gain the most from generative AI

#16

Post by version_five »

camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

I've had a few sessions recently with chatgpt (4) trying to use it instead of SO to help with coding. Your estimate sounds reasonable, probably +/- 25%.

It seems to be better at helping me with stuff I don't know anything about and that has more flexibility in the results. I got it to make a website for me (something I don't know how to do) and it did ok, though required a lot of back and forth, and it picked a lot of defaults, meaning if I'd had more rigid specifications it would have been harder.

I tried to get help in writing a pretty simple python function (some conditional formatting of text) and quickly realized would have been much better looking up what I wanted on SO and writing something myself than get caught in an awkward cycle of taking it's versions which had missed various requirements and trying to get it to correct then.

Anyway it seems to be best at giving minimal examples of things I don't know about, which is helpful. I'm still trying to find the right balance between when I should use it and when I'm better doing things the "old fashioned" way.

Re: Workers with less experience gain the most from generative AI

#17

Post by larve »

skepticATX wrote: post #8
camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

The funny thing about the 10x claims (I've even seen folks say 100x) is that results like these would be readily apparent. Even a broad 1.5x productivity increase would be world changing. And yet, we clearly are not seeing this currently. I'm not saying it will never happen, just that it will be very obvious if it does. 5-10% sounds about right to me.

The problem with these claims is that you can't really quantify a "10x" change. I have found a lot of emergent benefits to using LLMs. For example, because my cognitive load of wrangling APIs and understanding and refactoring legacy code and all the other nonsense of my day to day can be so heavily delegated, I actually feel refreshed at the end of the day, and can bang out a decently chunked feature on my opensource software on the couch (admittedly also boilerplate heavy code).

This means that I went from 0 opensource commits to 4000 since chatgpt came out.

Not just that, but I've gotten not only more adventurous, but have the time to consider doing drastic refactors and spend much more time thinking about my software.

I won't call it 10x or 100x, because that wouldn't mean anything, but surely it is a paradigm shift for me, completely world changing.

Re: Workers with less experience gain the most from generative AI

#18

Post by Swizec »

version_five wrote: post #16
camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

I've had a few sessions recently with chatgpt (4) trying to use it instead of SO to help with coding. Your estimate sounds reasonable, probably +/- 25%. It seems to be better at helping me with stuff I don't know anything about and that has more flexibility in the results. I got it to make a website for me (something I don't know how to do) and it did ok, though required a lot of back and forth, and it picked a lot o…

> it did ok, though required a lot of back and forth, and it picked a lot of defaults

This is what working with an expert looks like. They can’t read your mind.

However unlike an LLM, an expert knows what questions to ask to help you express what you want. This is currently lacking in AI, it doesn’t know how to guide you towards a solution you can’t verbalize because you are not the expert and don’t know what to specify.

Re: Workers with less experience gain the most from generative AI

#19

Post by mdp2021 »

As others are saying/hinting, workers with less ability to vet outputs may overlook crucial errors.

Take for example this automated help that I found only minutes ago - and seems to prove that generators can be racist (...I am joking):

https://ibb.co/v3VRTQQ

Re: Workers with less experience gain the most from generative AI

#20

Post by ignoramous »

It could be that folks who perhaps aren't too married to their current workflow are adopting to this newer paradigm faster?

For example, for places where English isn't the first language, LLMs are widely used for content-based tasks (posts, emails, support), because waiting 10s for Bard or GPT to output something higher quality is worth their time. Perhaps these folks end up being more attuned than usual, more in harmony and sync with the LLMs?

Speaking from experience, it took a while for me to move from vim to vscode, from Javascript to Typescript. We probably prefer our silos and feel uber comfortable in existing setups.

Post reply on HN