Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

231–240 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#231

Post by visarga »

henearkr wrote: post #179
quotemstr wrote: post #152

Earlier quoted context omitted.

If you believe in consciousness divorced from flesh, you're in strong intellectual company, but that's not a path I can go down. Metaphysically, I just can't accept the idea that the mind is anything but the computation performed by the brain.

I rather agree with you, but the interaction of the mind with the physical reality is also extremely important to shape it. GPT3 has no interaction with a physical world. Any formal system that cannot interact with something outside of itself will be intrinsically limited, for one thing by Gödel incompleteness theorem.

I agree about the need for an environment. The difference between GPT-3 and an agent in an environment is that GPT-3 only saw tons of static text, while an agent can design an action (an experiment), act it out and observe the results, drawing conclusions. Thus it can act in a way similar to the scientific method.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#232

Post by jengleton »

oehtXRwMkIs wrote: post #167
quotemstr wrote: post #150

Earlier quoted context omitted.

I have to disagree with this perspective and reiterate my original position: language is sentience. And to be fair, reasonable people stake out positions on both sides of this debate: I'm not claiming that the alternative proposition is somehow unreasonable. It's a legitimate subject of scholarly disagreement. Nevertheless, I'm still firm on language. Why? Because all complexity is ultimately about symbolic manipulat…

I think you're mixing up sentience/consciousness/intelligence. Many animals are sentient for example but as far as we know, they don't really have language. But I think I get what you're getting at, which I believe is "human level intelligence requires langauge". I think that's a reasonable take. But you said "sentience", and you said "is", which makes your position difficult to agree with.

Hold up, you can't just throw out a claim like "many animals are sentient" as if it's a statement of fact. You might be right, but there's a reason that "the hard problem of consciousness" is hard. We don't really have any way to distinguish sentience/non-sentience based on behavior. The whole concept is extremely mushy.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#233

Post by 6nf »

thewarrior wrote: post #164
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

How many attempts did it take or did you just choose the first one?

I have to admit, this is passing my turing test...

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#234

Post by jengleton »

jodrellblank wrote: post #102
quotemstr wrote: post #50

Earlier quoted context omitted.

And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.

> I don't believe that there's any experience of consciousness distinct from the use of language. Not sure there's one I can communicate to you, but I'm perfectly capable of forgetting the word for something and still knowing unambiguously yet wordlessly what it is, that's an experience. Catching a ball? Running? Experiencing emotions from wordless music? Viewing scenery? Engaging with a computer game? How are they n…

Also interesting to note that a good portion of people lack an internal monologue. This interview made the rounds a while back: https://www.youtube.com/watch?v=u69YSh-cFXY

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#235

Post by visarga »

tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

If society put a lid on OpenAI do you think others are going to stop doing language model research? Luddite responses don't fare well with new tech. Society will adapt, there is already a lot of garbage text online, and we'll find uses for LMs.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#236

Post by smallnamespace »

yomly wrote: post #169
smallnamespace wrote: post #46

Earlier quoted context omitted.

> no-one can train GPT-3 in any acceptable time except Gwern, who has spent probably as much time with GPT-3 and GPT-2 as any 'amateur' out there, is publicly out there saying that for most use cases, GPT-3 + creative use of prompts gets you better results than GPT-2 with finetuning . That's an amazing capability that Gwern elaborates on more here: > A new programming paradigm? The GPT-3 neural network is so large a…

I don't really understand your post (and don't know much about GPT3) are you suggesting that the model is stateful in that it can continue learning from successive prompts? Maybe you can elaborate on this: >instead, you interact with it, expressing any task in terms of natural language descriptions, requests, and examples, tweaking the prompt until it “understands” & it meta-learns the new task based on the high-leve…

The model is not stateful, but you can emulate state (certainly with GPT-3, but also with other language models) by simply feeding back earlier output.

For example, to simulate a chatbot, you start with a prompt. You then successively feed longer and longer chunks of the full chat back to the model, taking incrementally generated lines as the new AI's reply.

This is essentially how some of the 'use GPT-2 as a chatbot' front ends work in the world. This is also extended to make things like AI dungeon work: you can force the model to keep context within its attention by providing a good summary in the prompt.

To speculate a bit on why this seems to work, these models are massive and have read millions of texts in their corpus. Instead of 'retraining' on text which the model probably has already seen, the prompt is nudging the model to identify where in its on weights its encoded the knowledge before.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#237

Post by visarga »

simonsarris wrote: post #80

I could tell this was GPT-3 because it reads like SEO-style sentence structure :/ > I imagine that similar results can be obtained by republishing GPT-3’s outputs to other message boards, blogs, and social media. I actually wrote a bit about this scenario and how it could explode comment sections to stonewall topics, which I'm calling Commentdämmerung: https://simonsarris.com/commentdammerung Still, I think a lot of…

> You can't yet claim to have a magic die if you keep rolling it until you get the answer you want!

The infinite monkey theorem states that a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will almost surely type any given text, such as the complete works of William Shakespeare.

But here it was just 10 tries, not infinite.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#238

Post by geofft »

6nf wrote: post #233
thewarrior wrote: post #164

Earlier quoted context omitted.

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

How many attempts did it take or did you just choose the first one? I have to admit, this is passing my turing test...

Really? I got about halfway through and realized that the comment had no point. If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary.

Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#239

Post by visarga »

kristopolous wrote: post #115

It reads diffusive, unfocused and meandering, like a really bad writer who knows how to spell well and form proper sentences. All these AI generated systems have the same smell so far - from the ones that create art, music, literature - a convincing imitation of a lousy practitioner. This is fine for the background ... suspenseful music for a television show for instance or some descriptive balls-and-strikes style jo…

You're commenting on a rapidly evolving field as if it were static and we could imagine the results of the future models.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#240

Post by exolymph »

sgibat wrote: post #210
exolymph wrote: post #134

Earlier quoted context omitted.

Have you tried cowriting with it? Incredible for first drafts.

What's that process like? How do you use it when writing drafts?

I intentionally didn't participate super actively in this demo, but it should still give you the idea: https://www.youtube.com/watch?v=rZZ_gDjGkzM
Post reply on HN