Live data from Hacker News

GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

arxiv.org

121–130 of 235 posts

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#121
One major risk with this kind of AIs is that for the last few decades the mental image of what such AIs will be like (both by scientist and science fiction) has a _extreme serve_ mismatch with how they turned out.

In the past humanity people thought of them as this non-emotional purely logic driven beings and in turn problems people imagined would be such as this logic not considering emotions and emotional well being of humans or blindly (but using logic) pursing a specific goal no matter the consequences or not valuing freedom or gaining emotion. But in all that it still follows logic.

But now we have AIs which could have all the problems above _but doesn't use logical thinking_ to _archive goals_. Instead it uses complex overlapping _statistical models_ to _tell a believable story_ where believable is defined by the training data which is _widely inconsistent, wrong, misleading, discriminating, emotionally charged, etc._ because it's just scrapped from the internet. So there is _no systematic finding of goals, subgoals, plan etc_, there is _no logic_, the concept of "truth" simply _doesn't exist_ for such systems etc.

At the same time this turned out to be "often times" good enough to be usable for many task and can be convincing enough to make people believe that it's sentient.

But this also means it will retell common false information, misconceptions, discrimination, hatred etc. from the internet.

Similar it will do what people call "hallucination" and "lying" but it _not_ either of that and calling it that is misleading. Because it just doing _exactly_ what it was created for: Telling a "believable" story given the training data.

And gaslighting, misleading people and lying are a extremely deep ingrained part of the internet, i.e. a deep ingrained part of the data on which it bases what is "believable".

And while we can add tones of bandaid on top to try to hide/filter out such "bad" responses IMHO without fundamentally either changing the training data or the approach this is bound to fail while even stronger upholding a misleading illusion.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#122
post #90

Earlier quoted context omitted.

You should never medicate yourself after using a LLM, not in 2023 anyway! You should not run any code you have not vetted, or take any pills you don't know what they do.

Thats the problem, most people do take pills without knowing how they work, because doctor told them to take them. Also no one (probably) is vetting all the code they run on their computers. So this advice is kinda victim blaming. You ran the code that you did not verify? too bad, your own fault. You took advice from LLM? too bad, your own fault. You got tricked in the street by a crook? Too bad, your own fault. So w…

No, I verify the code I generate with chatGPT or Codex. I don't verify code I download.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#123
post #46

Earlier quoted context omitted.

For this specific example, an LLM in front of NICE will produce the correct result. It's a matter of time before this case will be fixed. From a non-expert's perspective through, an LLM is very dependable, unless it completely goes off the rail. How would anyone know when to trust it and when to be skeptical?

Trust but verify. We’ll get processes built around this for accuracy sensitive applications. I imagine it will look something like a GAN configuration with two or more LLMs trained to adversarially critique and fact check the outputs of the other(s). Might even mimic the relationship between the hemispheres of our brains.

Two models trained from different lineages won't hallucinate the same. When you want to check for hallucination the cost is to run the task on two models. For now. But soon it looks like LLMs will be better calibrated. It seems they are well calibrated after pre-training but become less calibrated after fine-tuning and RLHF. The last stage breaks its abilities to estimate confidnce scores correctly.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#124

Earlier quoted context omitted.

ChatGPT is nothing more than long-form autocomplete. The specifics of approach are irrelevant.

I knew something changed after AlphaGo. Compute could do what we thought only true intelligence can do. So I agree, LLMs are not a black swan. It will change everything, nevertheless. I keep reading these opinions that LLMs are just doing some advanced form of copy paste. Actually, we don't know what they are doing. Are they actually doing some form of modelling and abstraction? Seems likely to me.

AlphaGo is basically the same as image recognition, not sure why you thought AlphaGo was special compared to detecting faces. AlphaGo works the same way but maps game structures on a Go board instead of mapping facial structures and comparing those to known people.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#125
post #91

Earlier quoted context omitted.

LLM emergence was unpredictable in 2017. Even for the best AI people.

ChatGPT is nothing more than long-form autocomplete. The specifics of approach are irrelevant.

next bit prediction, or long-form autocomplete, may lead to general intelligence. http://mattmahoney.net/dc/rationale.html

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#126

Earlier quoted context omitted.

ChatGPT is nothing more than long-form autocomplete. The specifics of approach are irrelevant.

I knew something changed after AlphaGo. Compute could do what we thought only true intelligence can do. So I agree, LLMs are not a black swan. It will change everything, nevertheless. I keep reading these opinions that LLMs are just doing some advanced form of copy paste. Actually, we don't know what they are doing. Are they actually doing some form of modelling and abstraction? Seems likely to me.

"I keep reading these opinions that LLMs are just doing some advanced form of copy paste. Actually, we don't know what they are doing. Are they actually doing some form of modelling and abstraction? Seems likely to me."

This is exactly the problem with AI. For business or government, the answer is as important as the methodology employed. A black box does not work for the majority of use cases.

Until it can show its work, it's a sideshow.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#127
post #79

This is why my anxiety is at levels unseen before. I am a programmer who’s on a working visa. Assuming it does take my job, I have no idea what to do next… other than panic.

Panic seems like the wrong response. Figure out what you do that these things can’t. Use them to make yourself more productive. In the late 90s I was told my job was going to be outsourced, then dotcom, then the GFC, and lots of smaller bumps along the way. Those were actually somewhat scary times. Now should be excitement about what’s possible.

While you are not wrong, the idea that gives me anxiety is that this replaces all programmers. My job theoretically is safe for now. I work on fairly novel stuff, while a lot of my day is json parsing there is a big chunk of it that is domain specific. That said... it's that one day it will just take over all of programming that has me in dread. The only option then in my somewhat middle age is to unironically "learn a trade" I guess?

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#128
post #59

Earlier quoted context omitted.

Confining an LLM to the very narrow domain of "calculators" is a mistake, I think. You wouldn't say "a programmer that is 99% correct is worthless, I need 100%". I'm pushing it, but for a more fair comparison I'd say measure it against a programmer. How often are we wrong? 75% of the time? :) being generous here. It's the tools that make us productive. I don't know about you specifically, but I don't think you'll be…

> You need iteration and I believe these kinds of AI have the same issues as us. It's funny how we resort to humanizing the machines when their results are inaccurate. We don't do that with the calculator, because it's expected to be 100% bug free. When there's a bug in the calculator code we expect it to be fixed, not gradually improved. Speaking of bugs: mistakes in code is one thing, wrong output because of a fund…

Literally LLMs get much better with chain of thought, feedback, and/or consensus.

Gpt-3 performance on MultiArith goes from 18% to 92% with all three. This isn't some hackneyed anthropomizing. Countless research papers showing massive improvement with these processes.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#129

With little kids I am increasingly conscious about how to best equip them for the world when they are adults. I think previous playbook will need to be redrafted. 'Resilience' and 'Critical Thinking' are two things I am thinking are key any others?

I'm teaching mine gardening and jiu-jitsu.

Re: GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of LLMs

#130
post #2

==Occupations with no labeled exposed tasks== Agricultural Equipment Operators Athletes and Sports Competitors Automotive Glass Installers and Repairers Bus and Truck Mechanics and Diesel Engine Specialists Cement Masons and Concrete Finishers Cooks, Short Order Cutters and Trimmers, Hand Derrick Operators, Oil and Gas Dining Room and Cafeteria Attendants and Bartender Helpers Dishwashers Dredge Operators Electrical…

I feel like everyone is trying to draw very linear conclusions about what LLMs will affect, almost as if it's a circle that will be cut out of the market. But if we really get to the point where programmers can be replaced with LLMs, I expect something closer to swiss cheese: For example, if you're a meat packer, maybe you work someplace that doesn't want to pay for a fancy new machine that can do your job... But a m…

The other factor is unemployed office workers competing for the remaining jobs and thus driving down wages.
Post reply on HN