Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is GPT-3? written in layman's terms

tinkeredthinking.com

Post reply on HN
Print view View original

21–30 of 70 posts

Re: What is GPT-3? written in layman's terms

#21

Post by innagadadavida »

solidasparagus wrote: post #17

I have some issues with how this article describes GPT and its impact. The scenarios that are described as using GPT-3 remains the stuff of science fiction as far as I know. They would require major technological breakthroughs in areas of autonomous agents, combining language models with 'concept-based' models, and more mundane things like labelling (can you imagine how difficult it would take to accurately label all…

A much more realistic and accurate take on the tech than the article. One combination that can work is to have GPT3 with an unskilled human judge in the loop. The human can discriminate with common sense while the machine generates good jargon with correct grammar.

Re: What is GPT-3? written in layman's terms

#23

Post by SaltyLemonZest »

codingslave wrote: post #11

A comment from r/machinelearning: "It seems obvious from the demos that GPT-3 is capable of reasoning. But not consistently. It would be critical, imo, to see if we can identify a pattern of activity in it associated with the lucid responses vs activity when it prodcues nonsense. If/when we have such a apattern we would need to find a way to enforce it to happen in every interaction" And people agree: "Dunno why you…

I'm not convinced that "capable of reasoning, but not consistently" is a meaningful claim. The examples seem to primarily consist of people spending hours trying things, until eventually GPT-3 outputs a chunk of reasoning they could personally do in seconds. Does that mean that GPT-3 is doing the reasoning, or does it mean that GPT-3 is an English-based lookup table and they managed to find a clever sequence of search keys?

The fact that there could be reasoning going on is certainly exciting by itself. But I don't think it's fair to call it obvious without a compact specification for how to make GPT-3 perform a general class of reasoning. Less "here's a script to make it output stuff about balanced parens", more "here's a strategy to teach it most basic string manipulations".

Re: What is GPT-3? written in layman's terms

#24

Post by SaltyLemonZest »

jcims wrote: post #5

One thing I will say after seeing GPT-3 running live is that somehow the examples that you see trotted about in various articles never really seem to capture the spirit of the thing. Interacting with it directly and exploring the universe behind the prompt is one of the more profound computing experiences I've ever had.

Agreed. I've been playing AI Dungeon with their GPT-3 model, and it really does feel like there's a scatterbrained but human DM on the other side.

Re: What is GPT-3? written in layman's terms

#25

Post by Bx6667 »

solidasparagus wrote: post #17

I have some issues with how this article describes GPT and its impact. The scenarios that are described as using GPT-3 remains the stuff of science fiction as far as I know. They would require major technological breakthroughs in areas of autonomous agents, combining language models with 'concept-based' models, and more mundane things like labelling (can you imagine how difficult it would take to accurately label all…

Your dismissive use of the word terminator only makes it more difficult to accept your argument. You probably would have confidently predicted that the direct evolutionary descendants of humans were nowhere anywhere near becoming intelligent.

Re: What is GPT-3? written in layman's terms

#26

Post by pontifier »

bozzcl wrote: post #14
CharlesW wrote: post #9

Earlier quoted context omitted.

> Does that mean GPT-3 was trained on an arbitrary, huge database of text? Yes, 500 billion words, for a model with model 175 billion parameters.

So that continues my question: how much of that text would be on the private domain? If there is, is it all legally purchased?

I wonder if that's one reason they can't open it up. Perhaps they are ensuring that responses that come from the AI are sufficiently different from anything in the training corpus, so it can't be queried for sensitive data or large chunks of copyrighted material.

Re: What is GPT-3? written in layman's terms

#27

Post by SaltyLemonZest »

Bx6667 wrote: post #25
solidasparagus wrote: post #17

I have some issues with how this article describes GPT and its impact. The scenarios that are described as using GPT-3 remains the stuff of science fiction as far as I know. They would require major technological breakthroughs in areas of autonomous agents, combining language models with 'concept-based' models, and more mundane things like labelling (can you imagine how difficult it would take to accurately label all…

Your dismissive use of the word terminator only makes it more difficult to accept your argument. You probably would have confidently predicted that the direct evolutionary descendants of humans were nowhere anywhere near becoming intelligent.

The terminator idea is worth dismissing in this context. Even if GPT-3 is indeed intelligent, there's no reason to think that it's a goal-based system that could independently want things or do things.

Re: What is GPT-3? written in layman's terms

#28

Post by Trasmatta »

pontifier wrote: post #26
bozzcl wrote: post #14

Earlier quoted context omitted.

So that continues my question: how much of that text would be on the private domain? If there is, is it all legally purchased?

I wonder if that's one reason they can't open it up. Perhaps they are ensuring that responses that come from the AI are sufficiently different from anything in the training corpus, so it can't be queried for sensitive data or large chunks of copyrighted material.

It's basically already open through AI Dungeon. I've had all sorts of conversations so far with copyrighted characters.

Re: What is GPT-3? written in layman's terms

#29

Post by solidasparagus »

Bx6667 wrote: post #25
solidasparagus wrote: post #17

I have some issues with how this article describes GPT and its impact. The scenarios that are described as using GPT-3 remains the stuff of science fiction as far as I know. They would require major technological breakthroughs in areas of autonomous agents, combining language models with 'concept-based' models, and more mundane things like labelling (can you imagine how difficult it would take to accurately label all…

Your dismissive use of the word terminator only makes it more difficult to accept your argument. You probably would have confidently predicted that the direct evolutionary descendants of humans were nowhere anywhere near becoming intelligent.

There is zero evidence that GPT-3 has the ability to become self-aware and define new goals for itself. But I will give you that if your idea of 'close' is hundreds of thousands to millions of years, then yeah, AGI might be 'close'.

Re: What is GPT-3? written in layman's terms

#30

Post by pas »

solidasparagus wrote: post #17

I have some issues with how this article describes GPT and its impact. The scenarios that are described as using GPT-3 remains the stuff of science fiction as far as I know. They would require major technological breakthroughs in areas of autonomous agents, combining language models with 'concept-based' models, and more mundane things like labelling (can you imagine how difficult it would take to accurately label all…

Synapses' strength and ANN weights are sort of similar, but the learning process is different (co-activation vs loss propagation). Plus brains have neuroplasticity, but ANNs usually have at least one fully connected layer so it can emulate that, sort of.

Also how the brain's visual circuit parses stuff and how a CNN encodes concepts are sort of similar.

But, that's it so far :)

Post reply on HN