Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Workers with less experience gain the most from generative AI

mitsloan.mit.edu

Post reply on HN
Print view View original

31–40 of 120 posts

Re: Workers with less experience gain the most from generative AI

#31

Post by calvinmorrison »

camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

It is not intelligent. Let's just accept it. Ask what happens if you go due east exactly from NYC in a boat. Eventually you'll hit land where does it hit? Totally collapses with false information. Of course you can ask what parallel NYC is at and ask the western most point of Europe at the same parallel and it'll maybe get it right.

Give it basic directions. I'm in my car turning right on spring garden onto Broad street. I Travel north 10 miles on the same road. Where am I? It has no clue

Re: Workers with less experience gain the most from generative AI

#32

Post by JimtheCoder »

dmarchand90 wrote: post #22
skepticATX wrote: post #8

Earlier quoted context omitted.

The funny thing about the 10x claims (I've even seen folks say 100x) is that results like these would be readily apparent. Even a broad 1.5x productivity increase would be world changing. And yet, we clearly are not seeing this currently. I'm not saying it will never happen, just that it will be very obvious if it does. 5-10% sounds about right to me.

I dunno computers should be world changing. And they are. Yet the lives of ordinary people have hardly improved and are arguably worse since the 1980s

Worse in what sense?

Computers have done amazing things for humanity. I would not want to go back to a computerless world...

Re: Workers with less experience gain the most from generative AI

#33

Post by pydry »

adamwk wrote: post #24
camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

This matches my experience. for me it’s really just a better google (but still not good enough to replace it). This includes all the caveats where if you’re truly inexperienced it’s difficult to distinguish the good answers from blogspam quality or confidently incorrect answers.

Similar to google. With fewer trust cues and more hallucinations.

Re: Workers with less experience gain the most from generative AI

#34

Post by godelski »

camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

The thing I find them best for is fuzzy searching. When you'd have a hard time googling something, you ask the LLM. The answer you get back might also be fuzzy, but often that can be defuzzed and then correctly googled or resourced. In this way it is highly effective. But that is reducing the amount of time searching for information.

People that are 5-10x more effective at hard skills like programming, well I'm just convinced they weren't a good programmer to begin with and are doing easy problems. Basically, anyone that says that is telling on themselves.

Re: Workers with less experience gain the most from generative AI

#35

Post by mdp2021 »

dmarchand90 wrote: post #22
skepticATX wrote: post #8

Earlier quoted context omitted.

The funny thing about the 10x claims (I've even seen folks say 100x) is that results like these would be readily apparent. Even a broad 1.5x productivity increase would be world changing. And yet, we clearly are not seeing this currently. I'm not saying it will never happen, just that it will be very obvious if it does. 5-10% sounds about right to me.

I dunno computers should be world changing. And they are. Yet the lives of ordinary people have hardly improved and are arguably worse since the 1980s

That has nothing to do with what the poster is saying. And if tools improve but not quality of life, that is because minds are not improving along tools - this has been noted for centuries.

Re: Workers with less experience gain the most from generative AI

#36

Post by nashashmi »

camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

> I can't help but feel we're still N breakthroughs away from having it really impact dev jobs.

We will always be N breakthroughs away. Why? Because as our uses of computers improve, our work will become ever more complex. And our efforts will become even longer shots than they were before.

Re: Workers with less experience gain the most from generative AI

#37

Post by pfisherman »

My experience building AI powered tools for scientists has been the opposite.

Other commenters have pointed out the difficulties users with limited domaine expertise having in parsing the output and distinguishing what is useful, novel, interesting from what is trivial or incorrect.

This slows down the iterative process of learning how to use the tool as well as refining queries because they don’t have a big enough database of ground truth knowledge to calibrate against.

That being said these can work well as educational tools to rapidly get non-experts to a point where they can participate in conversations with experts. In that regard one might say that the bulk of the value is in the non-expert users because they are far more prevalent than experts.

The biggest advantage I saw for the expert researchers was that they knew (1) which big questions to ask, and (2) how to break down those big questions into smaller, more precise questions.

Re: Workers with less experience gain the most from generative AI

#38

Post by patrick451 »

One big difference between googling and asking an LLM is that it tends to be much easier to judge the legitimacy of information from a google result through informal heuristics. This is because google just links to a website, and there are all sorts of other factors that you use to subconsciously rank how reliable information on a particular website is, that mostly have nothing to do with the raw information.

For example, we all have different weights for how much trust information we get from, e.g, Wikipedia, stackoverflow, high profile bloggers in your domain, conspiracy websites, listacles, blog-spam sites etc, marketing materal etc. If the info is on a social site, is highly upvoted? Is the info from an academic journal, arxiv.org, an academic blog, or a wordoc you downloaded from scribd [1]?

Yes, this is judging a book by it's cover, but it's a heuristic that tends to work well.

By contrast, LLMs present all information to you with the same confidence in the same homogeneous interface. There is no external context. So all your normal heuristics for judging reliability are broken. -

[1] https://xkcd.com/1301/

Re: Workers with less experience gain the most from generative AI

#39

Post by gochi »

camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

Echoing this experience.

Even that 5-10% boost is mostly just from how horrible Google has become. I used to be able to find the proper sources much faster, but now I might as well ask LLMs since even that wait to generate the answer is technically "faster".

Post reply on HN