Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

171–180 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#171

Post by throwawaygh »

hnuser123456 wrote: post #169
throwawaygh wrote: post #162

Earlier quoted context omitted.

Gary Marcus doesn't claim that they claim that it's an AGI, so I really have no god damn clue what y'all are going on about here. Clearly, the point I'm making in the top comment is the following: OpenAI apparently restricts access to people -- even very highly respective scientists -- who happen to be critical of their previous work. It's impossible to prove intent, but limiting access for people like Gary Marcus is…

From a quick search on Gary, he's, in essence, spreading the word that GPT-3 isn't an AGI. Maybe this is useful to laymen to quench fears that it is one, but how should this garner a relationship with OpenAI? There are still people who have access who admit it's not, including OpenAI themselves already. I do wish the model was truly published openly, but despite it not being an AGI, it's already powerful enough to be…

> I do think there's an argument to be made that we should try to keep it in a good light to further interest in the field. How many more articles do we need like the one linked by OP?

This is how winter happens and is a decidedly anti-scientific mindset.

Do you consult with non-technical/low-technical execs on AI investments? If you did, the answer to your question would be obvious. For every dollar Microsoft and VC firms can spend on "we're the best technical minds in the world" Research&Marketing departments, there are thousands dollars in the rest of the economy that can only be captured by ensuring execs make prudent AI investments. Step zero is always deconstructing the hype around GPT/Alpha*/etc.

Re: GPT-3 has no idea what it’s talking about

#172

Post by Der_Einzige »

bra-ket wrote: post #137

The article is of course right but also a bit silly. Language models like GPT-X are producing grammatically correct sentences, along the lines of "Colorless green ideas sleep furiously". The NLP research more or less solved the old syntax problem using 'distributional semantics' but 'semantics' is a misnomer, it's all about syntax. In fact the most useful part of the article for me is that they mentioned Douglas Summ…

Gradient free optimization is not used much in Neural Networks except in Reinforcement Learning.

I think it's because backprop is objectively faster for most supervised problems than other techniques (e.g. simulated annealing or GAs)

Re: GPT-3 has no idea what it’s talking about

#173

Post by ssivark »

hackinthebochs wrote: post #115
ssivark wrote: post #86

Earlier quoted context omitted.

Statistics doesn’t preclude understanding, but statistics are definitely not enough. For example, uncertainties/probabilities/statistics is original to whether the model incorporates causal/reasoning structure. Any tractable amount of data with the former can’t approximate an ounce of the latter. All breakages will be attributed to “distribution shifts” of the underlying statistical distribution, or other pretty word…

>Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure]. I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. In a predictive language model, at some point the best way to model a text corpus that indirectly references the…

> I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner.

Yes, but suppose A implied B only if C were true. And in the training corpus C were always true (hence learned A=>B) but in the test corpus suppose C is not true, then the learned statistical rule is wrong.

The problem is that to cover all these possibilities will require not a billion documents but something like exp(billion) documents (exaggerating, but to give you a feel for combinatorial explosions) as @ClumsyPilot explains.

And problem is the real world always keeps “changing” (Eg: whether C is valid or not) compared to the examples you trained on. This is referred to in polite company as “distribution shift” where your underlying statistical distribution keeps evolving.

Re: GPT-3 has no idea what it’s talking about

#174

Post by im3w1l »

ClumsyPilot wrote: post #139
hackinthebochs wrote: post #64

Earlier quoted context omitted.

>If it did have a real understanding, prompt construction wouldn't matter as much This is only true if we assume GPT was never trained on satire or intentionally absurd text. But there's no reason to think this. Because it continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. If you treat GPT as a conversation bot and expect it to call you out when you give it a…

> if continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. Well then you can justify it outputting anything at all.

It's an autocompleter. If you want an answer to question, you have to make the answer the obvious (to a human) completion. If you want it to continue in a serious tone, you have to make sure the prompt is serious.

Re: GPT-3 has no idea what it’s talking about

#175

Post by Der_Einzige »

skatesor wrote: post #32
emteycz wrote: post #2

I got this as a reply (to an unrelated question) from PhilosopherAI.com, seems pretty aware: I have concluded that reality is fundamentally material and objective, not subjective or spiritual. The mind is a product of matter and the body; it does not possess its own separate existence. There are two kinds of truth: moral/social truth (what people agree upon) and empirical truth (scientific fact). The scientific metho…

You have to remember the AI cannot produce sentences or even words that someone else didn't already write. I'd totally agree it is 'aware' if it could meaningfully come to conclusions like these without getting them from someone else. You might say "don't all humans learn things from someone else" which is not really true because at some point there had to be a first person who learned something completely independen…

Uhhh, no. If your model is using subword tokenization like fasttext or... BERT with wordpiece (and the GPT-X models do this...), than you can generate entirely new words. Wasn't there a demo about doing exactly this a few days ago?

https://www.thisworddoesnotexist.com/

Re: GPT-3 has no idea what it’s talking about

#176

Post by azinman2 »

mordymoop wrote: post #108
azinman2 wrote: post #101

Earlier quoted context omitted.

Because when people say “AGI is near, just look at GPT-3,” it’d clear that we’re in a really good version of Searle’s chinese room. The lack of understanding is the important point.

I don’t recall any strong argument that Searle’s Chinese room can’t be an AGI, just that it can’t be conscious.

Certainly consciousness and understand are central to Searle’s argument. However from my perspective (particularly someone who is critical of all the recent DNN advances as being some harbinger of AGI), if all you’re doing is looking up replies from a dictionary, then you have no capacity to generalize, learn new things, adapt, empathize, have memory that isn’t already pre-computed and pre-allocated.

Now there are ways in which thru fine tuning etc that you can take GPT-3 and have it adapt in some way, thru such adaption or “context/attention networks” give it “memory,” in practice neither of these looks anything like AGI because right now these pages of this Chinese dictionary don’t have any relationship to each other. It becomes clear both in this article (you shouldn’t need some “correct” prompt to get it to make sense), and clear with long form generation that there isn’t a deeper understanding of the meaning of these words. I will say it’s very impressive what it can do when it does make sense, shockingly so, but we are very much in the position where we didn’t expect “chinese” to come out of the machine at all and thus we are projecting onto its outputs an anthropomorphisis that is unwarranted.

Re: GPT-3 has no idea what it’s talking about

#177

Post by azinman2 »

Isinlor wrote: post #112
azinman2 wrote: post #101

Earlier quoted context omitted.

Because when people say “AGI is near, just look at GPT-3,” it’d clear that we’re in a really good version of Searle’s chinese room. The lack of understanding is the important point.

To me you are also just a very good version of a Chinese room. How would you show me that you are not?

You can look up my comment history and find a continuity and a perspective mired in personal experience that a machine cannot (at least currently) replicate.

Re: GPT-3 has no idea what it’s talking about

#178

Post by throwawaygh »

lumost wrote: post #153
throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. This could realistically only occur if they cost-effectively fine-tune away the more egregious problems in beta - or convince enough investors that their next model with a 100 million dollar price tag will be able to handle something approximating AGI for realistic applications. This is…

~50-300 million sounds like an over-estimate, and the investment in GPT-3 is miniscule compared to SDC.

Plus, GPT-3 generates value even without direct product impact. I'm sure MSFT sales reps are already folding tons of nonsense about gpt-3 into their Azure pitches. Industry labs like OpenAI and DeepMind replace/augment the "Research & Development" model with a "Research & Marketing" model.

Re: GPT-3 has no idea what it’s talking about

#180

Post by rirarobo »

gweinberg wrote: post #132
pwinnski wrote: post #104

Earlier quoted context omitted.

Overfitting to the edge case is missing the point. I remind readers of this continuation: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, whi…

To be fair to the AI, stirring lemonade with a cigarette is so batshit insane that there really can't be a sensible continuation.

Hmm a sensible continuation to an absurd situation? Sounds like fun fiction.

> At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it promptly dissolved into my drink, creating a most unpleasant concoction, with an aroma which evoked memories of my grandfather's spittoon. I hoped no one had noticed, and promptly pocketed the cancer stick's wilted husk as discretely as I could manage. As I drew deep breaths from my potion's vapors, my mind was drawn into his small smoky study, and his form collected itself accross from me in his favorite leather lounge chair. I fell deeper into my nostalgia, pondering the unusual citrus scent in the air. Then, suddenly, I was ripped away from my reminiscing and pulled back to reality as a voice came calling, "sorry, can I help you?". I turned to see a tall handsome man, with ashy hair and sharp features, standing before me. It felt surprising intimate somehow, with just the two of us in the room now, while muffled sounds of other partygoers echoed from the house. The question's presentation seemed innocent enough at first, but I began to suspect that my faux pas had been found out. "Oh, thank you, not at all, I was just admiring the kitchen, beautiful craftsmanship," I replied, hoping my mundane alibi would suffice. "Oh I see, yes, we just recently renovated, what a nightmare," he said jovially. My hope for relief was extinguished as I realized who I was talking to, and I dove into my drink to bide my time. My face contorted, battling the horror, disgust, and protests of my tongue's unprepared palate. I could not think straight. My despair only deepened as I realized that in my lemonade-stirrer-seeking-mania I had picked that cursed stick from a pack adjacent the sugar bowl on the countertop, the same pack that the man in front of me was now fastidiously inspecting. How could I escape ...

Post reply on HN