Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

arxiv.org

Post reply on HN
Print view View original

81–90 of 235 posts

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#81

Post by shanebellone »

textninja wrote: post #75
dragonelite wrote: post #63

Earlier quoted context omitted.

Thats why im not falling for the hype again, i think i have seen like 2 previous AI hype cycles and all those fell off after like 3 months. Same with full automatic driving stuff.

This is a black swan event which, if anything, vindicates the hype from dreamers who were ahead of their time. They had a lot of the theory right but lacked the horsepower. This will complete the last mile for other AI tech and be the lacquer that gives it the all important finishing touch.

"This is a black swan event"

By definition, black swan events are unpredictable. This example isn't that.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#82

Post by rl3 »

ramraj07 wrote: post #65
sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

Not sure if you tried GPT-4 but my experience with 4 is quite different. It has been quite bullshit free, though not completely. For example I asked it to contrast oral and injectable semaglutide formulations. It did a bang up job. One thing I always do is ask it for evidence. And then I look the references up. Sometimes the references don’t say exactly what it said they will. I come back and have a discussion with i…

This tracks with my experience as well. It's not perfect and can be a little frustrating at times, but the capability provided is extremely powerful to the point of being game-changing.

It can pretty much augment any workflow in a net-beneficial way, provided you properly account for its shortcomings.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#83

Post by geraneum »

zone411 wrote: post #30
geraneum wrote: post #9

> Our findings indicate that the importance of science and critical thinking skills are strongly negatively associated with exposure, suggesting that occupations requiring these skills are less likely to be impacted by current language models. Conversely, programming and writing skills show a strong positive association with exposure, implying that occupations involving these skills are more susceptible to being infl…

>Am I reading this correctly that the assumption here is that programming and writing skills aren't reliant on critical thinking? No, they're just listing some skills that have both negative and positive associations with exposure. I don't think they intend to make a statement about whether the skills themselves are correlated. It's possible for them to be positively correlated with each other, even if one is positiv…

> No, they're just listing some skills that have both negative and positive associations with exposure.

Sure, but if that's the case then the writing is poorly worded, because that's how it reads if you follow the logic in the sentence.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#84

Post by geraneum »

wolpoli wrote: post #46
sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

For this specific example, an LLM in front of NICE will produce the correct result. It's a matter of time before this case will be fixed. From a non-expert's perspective through, an LLM is very dependable, unless it completely goes off the rail. How would anyone know when to trust it and when to be skeptical?

That's not how LLMs work. Including the NICE, if it actually isn't already, will not guaranty a "correct" result. It will increase the chance that the response is directly coming from the training but there is no guarante. If you are interested in why this is the case you can read this [1] post from Stephen Wolfram on how ChatGPT and in general LLMs work. This might give some insight on how and when to to use it more effectively.

[1] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#85

Post by makapuf »

mojuba wrote: post #36
bulbosaur123 wrote: post #33

Earlier quoted context omitted.

Was it GPT4 or GPT3.5? It spews less bullshit with each new iteration. Matter of time really.

I don't want to have a calculator (or say bookkeeping software) that gives correct results most of the time but not always, and then hear from the developers that it will get better with each iteration. I need a calculator that is correct 100% of time, not even 99.999%, because otherwise I can't rely on it at all. In other words, the utility of a calculator that is correct only 99% of time is zero, since you can't ev…

Do you have colleagues, bosses or reports that are correct 100% or the time? 99.999%? I would love colleagues that are 99% accurate, I certainly am not unfortunately.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#86

Post by jfoutz »

drinfinity wrote: post #54
mojuba wrote: post #36

Earlier quoted context omitted.

I don't want to have a calculator (or say bookkeeping software) that gives correct results most of the time but not always, and then hear from the developers that it will get better with each iteration. I need a calculator that is correct 100% of time, not even 99.999%, because otherwise I can't rely on it at all. In other words, the utility of a calculator that is correct only 99% of time is zero, since you can't ev…

Confining an LLM to the very narrow domain of "calculators" is a mistake, I think. You wouldn't say "a programmer that is 99% correct is worthless, I need 100%". I'm pushing it, but for a more fair comparison I'd say measure it against a programmer. How often are we wrong? 75% of the time? :) being generous here. It's the tools that make us productive. I don't know about you specifically, but I don't think you'll be…

Is the typing the hard part? I’ll look up libraries and apis, pretty regularly. maybe an algorithm every few years.

Figuring out what’s wanted from me takes forever though.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#87

Post by geraneum »

coldtea wrote: post #58
gumballindie wrote: post #27

Earlier quoted context omitted.

You have to give openai credit - their marketing strategy and effort is amazing. There’s spam everywhere. Albeit those claiming their jobs are already being replaced or those claiming they’ve written entire apps using chat gpt are quieting down a bit. They’ve either been called out for their nonsense or they got ahead of themselves.

> You have to give openai credit - their marketing strategy and effort is amazing. There’s spam everywhere. If it was that easy, every new such initiative (whether AI or any previous domain) would have it, of all those that have access to funding. One reason there's "spam" everywhere, is that a lot of the spam is genuine interest. Like how Haskell and Rust get tons of coverage on HN with zero (or close) actual market…

But nowadays, you can profit off people's attention alone directly. This does not prove that there is no genuine interest. I am geniuenly interested myself.

But, there are financial incentives in generating a lot of sensational content, whether positive or negative about almost everything including AI, Rust, political issues, even scientific issues like climate or pandemics, etc.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#88

Post by WoodenChair »

makapuf wrote: post #85
mojuba wrote: post #36

Earlier quoted context omitted.

I don't want to have a calculator (or say bookkeeping software) that gives correct results most of the time but not always, and then hear from the developers that it will get better with each iteration. I need a calculator that is correct 100% of time, not even 99.999%, because otherwise I can't rely on it at all. In other words, the utility of a calculator that is correct only 99% of time is zero, since you can't ev…

Do you have colleagues, bosses or reports that are correct 100% or the time? 99.999%? I would love colleagues that are 99% accurate, I certainly am not unfortunately.

> Do you have colleagues, bosses or reports that are correct 100% or the time? 99.999%? I would love colleagues that are 99% accurate, I certainly am not unfortunately.

I see this analogy all the time in these comment sections but it's not a very good one. A person is not a tool. One of the great achievements of humanity, and in computing in particular, is that we make tools that are more accurate than we are.

I expect a hammer to deliver a hard forward blow 100% of the time. If one out of one hundred times it delivers a hard backward blow, I cannot use it on a job site due to risk of injury to the user. The same is true of a calculator being used for financial transactions. And the same is true of a LLM that would be used for drug interactions as discussed in this thread. We already have 100% accurate ways of pulling data from a drug database—it's called SQL. A tool that is not as accurate is in at least some ways a step backward, even if it's easier to use due to its natural language interface.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#89

Post by geraneum »

light_hue_1 wrote: post #73
geraneum wrote: post #70

Earlier quoted context omitted.

This is what they are even admitting to: Under "3.4 Limitations of our methodology" - "3.4.1 Subjective human judgments" > A fundamental limitation of our approach lies in the subjectivity of the labeling. In our study, we employ annotators who are familiar with the GPT models’ capabilities. However, this group is not occupationally diverse, potentially leading to biased judgments regarding GPTs’ reliability and effe…

For sure. I had to read the paper to discover it's trash. But if you read the abstract, it looks like they thoroughly assessed how GPT will impact many professions. The sentence "Using a new rubric, we assess occupations based on their correspondence with GPT capabilities, incorporating both human expertise and classifications from GPT-4." does not scream to me "We asked 5 random people with no expertise in either th…

I agree with you. This is not scientifically sound research. Reads more like a brochure to be honest.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#90

Post by visarga »

sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

You should never medicate yourself after using a LLM, not in 2023 anyway! You should not run any code you have not vetted, or take any pills you don't know what they do.
Post reply on HN