Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

arxiv.org

Post reply on HN
Print view View original

61–70 of 235 posts

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#61

Post by magicalhippo »

mojuba wrote: post #36
bulbosaur123 wrote: post #33

Earlier quoted context omitted.

Was it GPT4 or GPT3.5? It spews less bullshit with each new iteration. Matter of time really.

I don't want to have a calculator (or say bookkeeping software) that gives correct results most of the time but not always, and then hear from the developers that it will get better with each iteration. I need a calculator that is correct 100% of time, not even 99.999%, because otherwise I can't rely on it at all. In other words, the utility of a calculator that is correct only 99% of time is zero, since you can't ev…

Surely this is "just" a matter of teaching the LLM to recognize this is a job for say Wolfram Aplha and generate a query to it, then feed the response back to you?

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#62

Post by Vespasian »

gwoolhurme wrote: post #39

This is why my anxiety is at levels unseen before. I am a programmer who’s on a working visa. Assuming it does take my job, I have no idea what to do next… other than panic.

I decided that being paralyzed with fear isn't a great strategy so I'm going to subscribe to OpenAI tools for a while and see how they can fit into my workflow.

It feels "wrong" on a strangely emotional level but what is happening is gonna happen anyway.

On a brighter note: There are bound to be a lot of unfounded, hype-based Marketing claims that look reasonable during the rush but fall flat over time. It would be a first for humanity if there weren't

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#63

Post by dragonelite »

sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

Thats why im not falling for the hype again, i think i have seen like 2 previous AI hype cycles and all those fell off after like 3 months. Same with full automatic driving stuff.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#64

Post by sophiabits »

mojuba wrote: post #36
bulbosaur123 wrote: post #33

Earlier quoted context omitted.

Was it GPT4 or GPT3.5? It spews less bullshit with each new iteration. Matter of time really.

I don't want to have a calculator (or say bookkeeping software) that gives correct results most of the time but not always, and then hear from the developers that it will get better with each iteration. I need a calculator that is correct 100% of time, not even 99.999%, because otherwise I can't rely on it at all. In other words, the utility of a calculator that is correct only 99% of time is zero, since you can't ev…

I suspect that humans have an accuracy lower than 99.999%, and are similarly capable of producing confidently incorrect results.

GPT has a lot of hype and hysteria around it, but demanding 100% accuracy from it is a bit over the top imo. It doesn't need to have 100% accuracy on any arbitrary prompt in order to be a useful and valuable tool.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#65

Post by ramraj07 »

sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

Not sure if you tried GPT-4 but my experience with 4 is quite different. It has been quite bullshit free, though not completely. For example I asked it to contrast oral and injectable semaglutide formulations. It did a bang up job. One thing I always do is ask it for evidence. And then I look the references up. Sometimes the references don’t say exactly what it said they will. I come back and have a discussion with it. It’s a back and forth for sure, but it’s an order of magnitude more informative and at least as accurate as a google search result is for me. I come out of it learning much more than I wanted to every single time. It’s pretty much part of my daily work and life, and I spend hours on it hitting the api limits now. So I’m not sure what others are telling about how it’s still shit.

If this AI system progresses no further, it’s already transformative. It’s an intelligence multiplier for some (I’m squarely in that category, just not sure if it’s 2x or 5x), but clearly for a lot of others it’s going to be something that takes away their livelihood.

I already use it for my coding work, one-way only, since I can’t paste proprietary code into it yet. The day something gpt-4-smart can plug into my orgs codebase, each of us will get at least 2x more efficient, conservatively.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#66

Post by Vespasian »

bulbosaur123 wrote: post #35
huuhee3 wrote: post #15

Earlier quoted context omitted.

Mass unemployment without adequate social safety net would be unfortunate.

Unfortunate in short term, but beneficial in long term.

How so?

It seems like masses of people locked out of any economic growth and with no perspective are a recipe for desaster if you ask me. That's the foundation on which unrest and rebellions are built and those are usually not pleasant affairs for most people.

I'm not saying that it will come to this.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#67

Post by drinfinity »

mojuba wrote: post #59
drinfinity wrote: post #54

Earlier quoted context omitted.

Confining an LLM to the very narrow domain of "calculators" is a mistake, I think. You wouldn't say "a programmer that is 99% correct is worthless, I need 100%". I'm pushing it, but for a more fair comparison I'd say measure it against a programmer. How often are we wrong? 75% of the time? :) being generous here. It's the tools that make us productive. I don't know about you specifically, but I don't think you'll be…

> You need iteration and I believe these kinds of AI have the same issues as us. It's funny how we resort to humanizing the machines when their results are inaccurate. We don't do that with the calculator, because it's expected to be 100% bug free. When there's a bug in the calculator code we expect it to be fixed, not gradually improved. Speaking of bugs: mistakes in code is one thing, wrong output because of a fund…

That's (IMO) too narrow view of what a "machine" is. Complex machinery of any kind never is 100% correct and needs constant correction and maintenance. I still think approaching this as a "calculator" is awkward at best.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#68

Post by coldtea »

mjburgess wrote: post #45
jalapenos wrote: post #24

I think more parallels should be drawn with what we were doing before: Googling it. Perhaps it's because ChatGPT seemed to happen much more suddenly than Google became a programming resource, but we're using them in much the same way. Asking for pre-made solutions, explanations, troubleshooting tips etc. ChatGPT just does the job way better. But no-one was worried Google would put knowledge workers out of a job.

Everything I've used ChatGPT for so far I could, more-or-less, have written or searched for myself in the time it took to get it out of chatgpt correctly. And I've often just had to rewrite it completely. If you're an expert (writer, programmer, etc.) it's often faster to type it as-needed than modify chatgpt's output. If you're not then either it's not reliable enough, since you do not have the expertise to modify;…

>Everything I've used ChatGPT for so far I could, more-or-less, have written or searched for myself in the time it took to get it out of chatgpt correctly.

That's not even true. For one, it can stub a whole new function or coding project, intelligently, in a few seconds after the prompt, whether it's RoR or DSP code or whatever. For one unfamiliar with the domain, this can take hours or a full day, even with examples found on Google. Heck, even looking up and understanding how to use some command line flags in a shell pipeline can need lots of looking around, even if you have been using Linux/Unix for ages.

For a domain expert? They could do such things very fast. But several times slower still than GPT. Think minutes or half hour instead of seconds. Even the row typing and file creation would be some minutes.

It's also very premature when people judge a service we've had for like 5 years and has already changed by leaps and bounds, as if it's the peak stage, without considering what it could be in 5 or 10, or with different variants tuned for specific tasks.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#69

Post by fendy3002 »

sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

but it's very good for experts or those who're very knowledgeable and want to dig quick information / bootstrapping something, since they can digest whether the information is correct, useful, or wasteful.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#70

Post by geraneum »

light_hue_1 wrote: post #26

Total trash cloaked in a complicated story. What they actually did is ask 5 random people to rate what thought a language model could do to help different professions. These 5 random people don't know anything about the professions they're rating, just what anyone off the street knows, and they know as much about GPT as anyone who has briefly played with it. The title should have been "We asked 5 friends to see what…

This is what they are even admitting to:

Under "3.4 Limitations of our methodology" - "3.4.1 Subjective human judgments"

> A fundamental limitation of our approach lies in the subjectivity of the labeling. In our study, we employ annotators who are familiar with the GPT models’ capabilities. However, this group is not occupationally diverse, potentially leading to biased judgments regarding GPTs’ reliability and effectiveness in performing tasks within unfamiliar occupations. We acknowledge that obtaining high-quality labels for each task in an occupation requires workers engaged in those occupations or, at a minimum, possessing in-depth knowledge of the diverse tasks within those occupations. This represents an important area for future work in validating these results.

Post reply on HN