Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 Creative Fiction

gwern.net

Post reply on HN
Print view View original

21–30 of 103 posts

Re: GPT-3 Creative Fiction

#21

Post by InfiniteRand »

ixvvqktiwl wrote: post #9

I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching. But I get it, it's exciting, and it's an easy way to get VC money. Perhaps one day we'll get something useful aside from the various pattern matching applications (image recognition, speech to text, etc). I'm skeptical…

One thing that I find interesting is how children also essentially do advanced mimicry to fill in gaps of their knowledge, what would be interesting is to see a more traditional inference engine type approach that then used a GPT-3 style approach when the knowledge of a situation dipped beyond a certain level, and then take the response to the GPT-3 style approach as a datapoint for inference engine style reasoning

Re: GPT-3 Creative Fiction

#22

Post by mistercow »

ixvvqktiwl wrote: post #9

I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching. But I get it, it's exciting, and it's an easy way to get VC money. Perhaps one day we'll get something useful aside from the various pattern matching applications (image recognition, speech to text, etc). I'm skeptical…

Do you consider TabNine to be a "pattern matching application"?

Re: GPT-3 Creative Fiction

#23

Post by messe »

2bitencryption wrote: post #6

GPT's take on the navy seal copypasta, in the style of a KGB spy: "I have over 300 confirmed red scares." Haha, that is genuinely one of the funniest versions of that I've ever seen, human-generated or otherwise. That level of inference is really amazing.

Idk, Donald Trump's "I have over 300 confirmed bankruptcies" did it for me.

Re: GPT-3 Creative Fiction

#24

Post by sutterbomb »

ixvvqktiwl wrote: post #13
627467 wrote: post #11

Earlier quoted context omitted.

I'm not disagreeing with you but where's the frontier between "pattern matching" and "intelligence"? I also think human intelligence and creativity will always be judged by other humans as _better_, more genuine - as long as the judges can trust that given creation was lead by a human. Among creative types "derivative" is a derogatory label used frequently. "Lesser artists borrow, great artists steal" also has an imp…

That's a good point. I think we need to have a better understanding about how the brain works before we can properly answer that question. At this point there's a lot we don't understand about the brain, and psychology for that matter, it's basically a black box that we can observe through a microscope or other imaging system but we don't really understand what's happening. An ant only has around 250,000 neurons, yet…

The current leading theories on how the brain works are that yes, they are essentially very impressive prediction machines, which obviously rely on all sorts of pattern matching to make said predictions. You can look up Karl Friston's free energy principle or check this book for more details

https://www.amazon.com/Surfing-Uncertainty-Prediction-Action...

Re: GPT-3 Creative Fiction

#25

Post by sutterbomb »

api wrote: post #16

I'm gonna put forward the very view that gwern repeatedly argues against: "but... it's not understanding ." So far I see no evidence that this thing or anything else like it has any actual understanding, any model of the world. Indeed it can't as it possesses no sensory apparatus. It's not embodied. It doesn't experience anything. I'm not sure the OpenAI folks would argue with me, but it seems Gwern asserts that this…

What makes you confident that you aren't overestimating the importance that we "experience anything"

Re: GPT-3 Creative Fiction

#26

Post by visarga »

ixvvqktiwl wrote: post #9

I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching. But I get it, it's exciting, and it's an easy way to get VC money. Perhaps one day we'll get something useful aside from the various pattern matching applications (image recognition, speech to text, etc). I'm skeptical…

> I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best.

GPT-3's amazing ability to pick up what we want from prompts lift it above mimicry. Intelligence is about fast solving of novel tasks (with little supervision). GPT-3 does this more than any other language model.

Re: GPT-3 Creative Fiction

#27

Post by visarga »

627467 wrote: post #11
ixvvqktiwl wrote: post #9

I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching. But I get it, it's exciting, and it's an easy way to get VC money. Perhaps one day we'll get something useful aside from the various pattern matching applications (image recognition, speech to text, etc). I'm skeptical…

I'm not disagreeing with you but where's the frontier between "pattern matching" and "intelligence"? I also think human intelligence and creativity will always be judged by other humans as _better_, more genuine - as long as the judges can trust that given creation was lead by a human. Among creative types "derivative" is a derogatory label used frequently. "Lesser artists borrow, great artists steal" also has an imp…

> where's the frontier between "pattern matching" and "intelligence"?

I recently saw this great video related to your very question.

Yannic Kilcher: Paper review "On the Measure of Intelligence by François Chollet" https://www.youtube.com/watch?v=cuyM63ugsxI

Re: GPT-3 Creative Fiction

#28

Post by webmaven »

ixvvqktiwl wrote: post #9

I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching. But I get it, it's exciting, and it's an easy way to get VC money. Perhaps one day we'll get something useful aside from the various pattern matching applications (image recognition, speech to text, etc). I'm skeptical…

> I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching.

"Advanced Mimicry" is in fact an entirely apt description of a lot of human activity that falls under the heading of "intelligence", though not necessarily particularly "smart". So, we could call it "Artificial Stupidity" instead, if you like.

Re: GPT-3 Creative Fiction

#29

Post by p1esk »

sillysaurusx wrote: post #2

Fine tuning GPT-3 is one of the biggest challenges, because it's behind an API. The weights aren't available to researchers, so we can't make it do anything it doesn't already do. But, that's fair. It's OpenAI's weights; they can keep them locked up if they want to. What caught my attention, though, is that supposedly OpenAI is working on a way to support fine-tuning. If you think about the logistics of that, it's a…

Yes, I guess you can have the API to provide you the intermediate layer outputs, instead of the predictions. However, if you then want to finetune your own extra layers using these intermediate outputs as inputs, would the API be able to produce them fast enough for you to do the finetuning of your own layers in reasonable time? That's assuming the extra layers are located on your own servers. Or would OpenAI be willing to actually create the extra layers on their own machines and let you finetune those? In the second scenario, you would need to move your dataset to their servers.

Actually, since they used Azure cloud to train GPT-3, I don't see why they wouldn't just let you pay for spinning up Azure instances to train your extra layers, and connect those to the model.

Re: GPT-3 Creative Fiction

#30

Post by Reedx »

Carmack posted (yesterday) an interesting thought on models like GPT-3:

"Big AI models like GPT-3 train on massive internet text dumps, but the data is assumed to be independent and identically distributed. Incorporating time information for a decade of data might allow them to start writing tomorrow's reddit or twitter trends."

https://twitter.com/ID_AA_Carmack/status/1278840413919551488

Post reply on HN