Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

arxiv.org

Post reply on HN
Print view View original

71–80 of 235 posts

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#71

Post by stockboss »

mojuba wrote: post #36
bulbosaur123 wrote: post #33

Earlier quoted context omitted.

Was it GPT4 or GPT3.5? It spews less bullshit with each new iteration. Matter of time really.

I don't want to have a calculator (or say bookkeeping software) that gives correct results most of the time but not always, and then hear from the developers that it will get better with each iteration. I need a calculator that is correct 100% of time, not even 99.999%, because otherwise I can't rely on it at all. In other words, the utility of a calculator that is correct only 99% of time is zero, since you can't ev…

have you ever got on an airplane? are you aware that they are not 100% safe?

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#72

Post by gombosg »

coldtea wrote: post #51
Yoric wrote: post #31

Earlier quoted context omitted.

> So, a lot of engineers that are just doign teh same old trick: Writing HTTP endpoints, parsing json. Mapping data types.. Yes that could be automated. And to be fair, automation for all of that already pretty much exists.

And to be fair, even though it exists, there's a huge majority that is done manually, even though it needs zero or very little manual "creativity" between specification and implementation

I agree. Usually you're already working within some framework or DSL where you can describe what you want to do. Ideally, you already have an idiomatic codebase enabling you to succinctly transform specifications into code.

Let's take parent poster's issues:

> Writing HTTP endpoints, parsing json. Mapping data types.

The generative model (for now) won't figure out for you: authentication, authorization, input form schema, JSON schema, required & optional fields, field constraints, entity modeling, indexing, query optimization, just to name a few basic issues we are looking at when "just developing CRUD apps".

If any of those go bad, it would result in 400s, 500s, performance or security issues.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#73

Post by light_hue_1 »

geraneum wrote: post #70
light_hue_1 wrote: post #26

Total trash cloaked in a complicated story. What they actually did is ask 5 random people to rate what thought a language model could do to help different professions. These 5 random people don't know anything about the professions they're rating, just what anyone off the street knows, and they know as much about GPT as anyone who has briefly played with it. The title should have been "We asked 5 friends to see what…

This is what they are even admitting to: Under "3.4 Limitations of our methodology" - "3.4.1 Subjective human judgments" > A fundamental limitation of our approach lies in the subjectivity of the labeling. In our study, we employ annotators who are familiar with the GPT models’ capabilities. However, this group is not occupationally diverse, potentially leading to biased judgments regarding GPTs’ reliability and effe…

For sure. I had to read the paper to discover it's trash.

But if you read the abstract, it looks like they thoroughly assessed how GPT will impact many professions.

The sentence "Using a new rubric, we assess occupations based on their correspondence with GPT capabilities, incorporating both human expertise and classifications from GPT-4." does not scream to me "We asked 5 random people with no expertise in either these professions or GPT-4 what they thought and report those results".

This is borderline dishonest.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#74

Post by gwoolhurme »

Vespasian wrote: post #62
gwoolhurme wrote: post #39

This is why my anxiety is at levels unseen before. I am a programmer who’s on a working visa. Assuming it does take my job, I have no idea what to do next… other than panic.

I decided that being paralyzed with fear isn't a great strategy so I'm going to subscribe to OpenAI tools for a while and see how they can fit into my workflow. It feels "wrong" on a strangely emotional level but what is happening is gonna happen anyway. On a brighter note: There are bound to be a lot of unfounded, hype-based Marketing claims that look reasonable during the rush but fall flat over time. It would be a…

That is certainly true about the hype-machine it is in full force. I imagine that will level off soon and the really useful and novel stuff will float to the top. I have no backup plan but as you said, what happens is going to happen? It's a rollercoaster for me personally where some days I am in absolute dread and fear about the future. Then other days I just go along my day as usual.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#75

Post by textninja »

dragonelite wrote: post #63
sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

Thats why im not falling for the hype again, i think i have seen like 2 previous AI hype cycles and all those fell off after like 3 months. Same with full automatic driving stuff.

This is a black swan event which, if anything, vindicates the hype from dreamers who were ahead of their time. They had a lot of the theory right but lacked the horsepower. This will complete the last mile for other AI tech and be the lacquer that gives it the all important finishing touch.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#76

Post by textninja »

wolpoli wrote: post #46
sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

For this specific example, an LLM in front of NICE will produce the correct result. It's a matter of time before this case will be fixed. From a non-expert's perspective through, an LLM is very dependable, unless it completely goes off the rail. How would anyone know when to trust it and when to be skeptical?

Trust but verify. We’ll get processes built around this for accuracy sensitive applications. I imagine it will look something like a GAN configuration with two or more LLMs trained to adversarially critique and fact check the outputs of the other(s). Might even mimic the relationship between the hemispheres of our brains.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#77

Post by mjburgess »

coldtea wrote: post #68
mjburgess wrote: post #45

Earlier quoted context omitted.

Everything I've used ChatGPT for so far I could, more-or-less, have written or searched for myself in the time it took to get it out of chatgpt correctly. And I've often just had to rewrite it completely. If you're an expert (writer, programmer, etc.) it's often faster to type it as-needed than modify chatgpt's output. If you're not then either it's not reliable enough, since you do not have the expertise to modify;…

> Everything I've used ChatGPT for so far I could, more-or-less, have written or searched for myself in the time it took to get it out of chatgpt correctly. That's not even true. For one, it can stub a whole new function or coding project, intelligently, in a few seconds after the prompt, whether it's RoR or DSP code or whatever. For one unfamiliar with the domain, this can take hours or a full day, even with example…

> For one unfamiliar with the domain, this can take hours or a full day

Yes, I did say expert.

> But several times slower still than GPT.

Almost all the output needs to be re-read, modified and integrated in the project. This often takes longer than just typing it out, even from docs -- because you're still forced to think through the solution -- which is most of the time.

Typing is quick

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#78

Post by shanebellone »

sophiabits wrote: post #64
mojuba wrote: post #36

Earlier quoted context omitted.

I don't want to have a calculator (or say bookkeeping software) that gives correct results most of the time but not always, and then hear from the developers that it will get better with each iteration. I need a calculator that is correct 100% of time, not even 99.999%, because otherwise I can't rely on it at all. In other words, the utility of a calculator that is correct only 99% of time is zero, since you can't ev…

I suspect that humans have an accuracy lower than 99.999%, and are similarly capable of producing confidently incorrect results. GPT has a lot of hype and hysteria around it, but demanding 100% accuracy from it is a bit over the top imo. It doesn't need to have 100% accuracy on any arbitrary prompt in order to be a useful and valuable tool.

"It doesn't need to have 100% accuracy on any arbitrary prompt in order to be a useful and valuable tool."

Yet anything less than represents a danger. Never underestimate stupid.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#79

Post by matwood »

gwoolhurme wrote: post #39

This is why my anxiety is at levels unseen before. I am a programmer who’s on a working visa. Assuming it does take my job, I have no idea what to do next… other than panic.

Panic seems like the wrong response. Figure out what you do that these things can’t. Use them to make yourself more productive.

In the late 90s I was told my job was going to be outsourced, then dotcom, then the GFC, and lots of smaller bumps along the way. Those were actually somewhat scary times. Now should be excitement about what’s possible.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#80

Post by mjburgess »

bob1029 wrote: post #53
mjburgess wrote: post #45

Earlier quoted context omitted.

Everything I've used ChatGPT for so far I could, more-or-less, have written or searched for myself in the time it took to get it out of chatgpt correctly. And I've often just had to rewrite it completely. If you're an expert (writer, programmer, etc.) it's often faster to type it as-needed than modify chatgpt's output. If you're not then either it's not reliable enough, since you do not have the expertise to modify;…

> People seem to be forgetting that in the vast majority of cases the thing ChatGPT is giving you is also available on stackoverflow, github, wikipeida, or 101 other high quality online sources. No one is forgetting anything over here. With StackOverflow, I have to scroll through ~2 pages of bing/google trash to even begin looking at a potential solution. This is only the beginning of the process. 99% of the time, th…

Sure, but is turning 90min of googling into 5min + 30min of rewriting, "revolutionary" ?

When you're googling and reading, you're doing the work of building your understanding. That's shifted in the GPT case to afterwards.

It's an advance; it's not nothing. Does it reduce the need to actually understand what you're writing -- no. That's most of the time anyway

Post reply on HN