Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

arxiv.org

Post reply on HN
Print view View original

191–200 of 235 posts

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#191

Post by boringuser1 »

dboreham wrote: post #168
sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

This has been my experience: Chat GPT is astonishing in terms of producing plausible natural language text that looks related to the question. That's an astounding achievement imho. It does not (and I suspect wasn't expected to) produce correct answers to questions.

[dead]

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#192

Post by jalapenos »

giuscri wrote: post #43
jalapenos wrote: post #24

I think more parallels should be drawn with what we were doing before: Googling it. Perhaps it's because ChatGPT seemed to happen much more suddenly than Google became a programming resource, but we're using them in much the same way. Asking for pre-made solutions, explanations, troubleshooting tips etc. ChatGPT just does the job way better. But no-one was worried Google would put knowledge workers out of a job.

In the last days I'm using ChatGPT as my first choice when searching for something and then googling if I'm not sure GPT is not hallucinating. Are they able to sustain the load?

I imagine Microsoft is giving them any & all infra they need.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#193

Post by pwinnski »

dboreham wrote: post #168
sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

This has been my experience: Chat GPT is astonishing in terms of producing plausible natural language text that looks related to the question. That's an astounding achievement imho. It does not (and I suspect wasn't expected to) produce correct answers to questions.

I had an extended back-and-forth in which I asked ChatGPT how to undo mistakenly marking a message as junk in Messages on iOS.

It gave me a 7-step answer that was completely wrong. Then I said no, you can't do that, it apologized and gave me an 8-step answer that was completely wrong. When I pointed out in this case there is no "Junk" folder in iOS Messages, it apologized for the confusion and gave me another 8-step answer that was completely wrong. When I pointed out why that one wouldn't work, it gave up and said recovery was impossible and that I would need to contact the sender to re-send, and be more careful when marking messages as junk. This was still wrong, as recovery is possible, just not by any of the means it described.

So yeah, I have been super-impressed by the quality of output from these LLMs, but I cannot imagine actually relying on one for anything where correctness matters.

Giving me a nice list of Korean shoegaze bands, sure. Its step-by-step for how to become a better volleyball player will be great for my daughter, and was better than the answer to the same question from Google. But correctness? No.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#194

Post by kristjansson »

srg0 wrote: post #167

So, researches paid by OpenAI are going to publish a paper to convince everyone else that an OpenAI product could have some economic impact. Am I the only one who sees a conflict of interest here? Is this preprint already submitted for peer review or published somewhere? Or is it just an advertisement formatted in LaTeX?

It’s building on a pretty serious thread of research, and demonstrates reasonable agreement with prior work (table 9) so I don’t think it can be dismissed as an advertisement.

Generating exposure ratings via GPT-4, from annotations provided by OpenAI people does definitely put a positive bias on the exposure estimates (which they acknowledge)

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#195

Post by shanebellone »

BurningFrog wrote: post #179
shanebellone wrote: post #154

Earlier quoted context omitted.

This is a shallow counterargument that entirely misses the point.

This is not even an argument, only a disagreement.

"This is not even an argument, only a disagreement."

It's a straw man.

The need for transparency in process is known, documented, and undisputed. Your comment has no relevance. My brain might be a black box, but I can still communicate and/or document the specifics of a process.

Can [insert your preferred model] do that? Didn't think so.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#196

Post by deeviant »

maaanu wrote: post #142
quonn wrote: post #116

Earlier quoted context omitted.

I found it useful for writing unit tests, which really is a stupid brain dead code monkey task.

Really? For me it could not generate meaning full tests, even for the simplest react-components. E.g. it tried to test the display of an error-message in a field in a form. The error message was basically "this is field is required and can't be empty.". It tried to test that, by inserting valid data into the field...

To be fair, many human software engineers can't generate meaningful tests.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#197

Post by kristjansson »

light_hue_1 wrote: post #26

Total trash cloaked in a complicated story. What they actually did is ask 5 random people to rate what thought a language model could do to help different professions. These 5 random people don't know anything about the professions they're rating, just what anyone off the street knows, and they know as much about GPT as anyone who has briefly played with it. The title should have been "We asked 5 friends to see what…

Eh ... they report pretty good alignment with other studies on the topic, so there's at least some signal. Whether their labels contribute any new information is unknown, and the forecasts of any of the literature they cite are untestable (expect by the wait-and-see approach).

That said, some attempts at prognostication are preferable to a collective shrug, and people at OpenAI are better positioned than others to assess what GPT-4+ is (will be) capable of, while clearly under-equipped to map that capabilities to the intricacies of 1000 occupational categories.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#198

Post by Der_Einzige »

js8 wrote: post #105
kossTKR wrote: post #94

Earlier quoted context omitted.

You didn't use version 4 did you? Because if feel this is pretty much already out of date. 4 does this _a lot_ less! And 5, 6 or whatever will probably be better, so i don't even really get the point here.

I find there are two kinds of people in the world. First are saying that LLMs are bullshitting. The second are bullshitting about whatever LLMs are saying.

The duality of mankind. On one hand, we have skeptics asserting that LLMs are masters of malarkey. On the other, we've got those regurgitating the verbal gymnastics of said LLMs. But lo and behold, there exists a third breed: those who delight in deciphering the eloquent dance between the bullshitters and the bullshitted.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#199

Post by kristjansson »

malborodog wrote: post #49
sirsinsalot wrote: post #29

I asked Chat GPT which antacid medications are contraindicated for some medication I'm on. Easily found through NICE. It made up a severe risk of death taking a very common medicine combo. It was super convincing, even giving information on how long to avoid taking them together. It was pure bullshit. I think as much as hyping the benefits we need to hype the flaws and dangers. If the public at large learn to trust t…

This kind of problem is (trivially?) solvable through the ReAct framework, like LangChain etc. Basically you get good data, vector embed it, and make sure the LLM knows where to look for accurate information.

Who - WHO - thought that naming something new 'the ReAct framework' in 2023 was a reasonable choice.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#200

Post by pixl97 »

Jensson wrote: post #103
drinfinity wrote: post #67

Earlier quoted context omitted.

That's (IMO) too narrow view of what a "machine" is. Complex machinery of any kind never is 100% correct and needs constant correction and maintenance. I still think approaching this as a "calculator" is awkward at best.

> Complex machinery of any kind never is 100% correct and needs constant correction and maintenance Computers are extremely close to 100%, we generally expect a CPU to never make errors even after years of working. If it starts making any errors at all we throw it away and make a new one.

This is a very weird statement that's failing based on logical category.

My computer will pretty much add 1+1 correctly forever never making a mistake.

My computer will perform an 'error' every time I put bad code into it, and some of those logic chains and error conditions are not very obvious.

The issue here is you think the LLM is performing a category 1 error, when the problem we are seeing is a much more human like category 2 error.

Post reply on HN