Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

151–160 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#151

Post by thewarrior »

cs702 wrote: post #147
thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

The thing is that this thing has now crossed into the uncanny valley. Earlier it would have great trouble making a single sentence that makes sense. You only ever remember whether the last two sentences made sense and go together. And with GPT-3 any pair of sentences always makes almost perfect sense. By the time you're four sentences down you go wait a minute ...

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#152

Post by quotemstr »

pastrami_panda wrote: post #92
quotemstr wrote: post #50

Earlier quoted context omitted.

And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.

> I don't believe that there's any experience of consciousness distinct from the use of language. To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example. I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder th…

If you believe in consciousness divorced from flesh, you're in strong intellectual company, but that's not a path I can go down. Metaphysically, I just can't accept the idea that the mind is anything but the computation performed by the brain.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#153

Post by crispyporkbites »

Prediction: GPT-3 and it’s subsequent iterations will eventually take away Google’s dominance in Search. In fact we may not have text based search as we know it in 2030.

I could use this to churn out blog posts and content pages by the dozen on any and every subject, which is basically how search engines rank content these days.

Will Google ever be able to tell the difference between gpt-3 text and human text? Or will the results become garbage?

It’s a hard problem to solve, it has shades of the Halting problem by definition.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#154

Post by colordrops »

abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

> I'll be impressed the day I can see a program that can 1) only rely on its own limited experiential inputs

Hasn't the typical human taken in orders of magnitude more data than this example? And the data has been of both direct sensory experience and texts from other people as well.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#155

Post by CamperBob2 »

cs702 wrote: post #147
thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

Worse, Sokal's Revenge is probably inevitable, in which someone will generate a nightmarishly complex mathematical proof that takes the whole field in unexplored directions. Some of the most respected professors and most promising students will then be distracted for years trying, and ultimately failing, to make sense of it.

Some say this has already happened. Nobody has ever seen the Social Text editors and Mochizuki in the same room together, have they?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#156

Post by guhayun »

abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

1) It DOES rely on its own limited input,meta learning

2) Quite irrelevant,that's a motivation problem

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#157

Post by nullc »

Dangeranger wrote: post #131
nullc wrote: post #84

I've been using GPT2 as a writing aid for a little while (the 1558M model is a lot better than the smaller ones). Some of my recently highly upvoted HN comments were substantially written by GPT-2. There is quite a bit of art that goes into crafting prompts that produce useful output. There are a few things about GPT2 that I didn't like (for instance, the number of "quoted text" blocks) but I have found that if I put…

Can you elaborate on how you are having GPT2 contribute to your comments? What is your process?

I write a prompt, often copying text from the articles or other comments, and have it generate a lot of completions. I skim over the completions and grab interesting parts.

For example, complaining about "quoted text" in my above comment was GPT2's suggestion (and also an actual issue with GPT2 which it was exhibiting by producing that text). Actually, all the text above from "There are a few" and beyond were written by GPT2. They were true enough, though more often I just extract the ideas and write my own text.

In some contexts, but not on HN, I've also found it useful to get GPT2 to generate replies to my posts. It occasionally spots counter-arguments that I forgot to address and I can edit my comments to address them.

Using computer text generation as a writing tool isn't new to me, I wrote my first computer poetry generator in the mid-90s. I've put out a few other things which were mostly machine generated.

To be clear: GPT2 puts out a lot of junk-- mostly junk, in fact. But a lot beauty can be found sifting gems out of a sea of noise. Some authors have taken psychedelic drugs to enhance their creative processor, with GPT2 it is your word-processor that takes the drugs.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#158

Post by abernard1 »

exolymph wrote: post #149
abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

> This does not impress me in the slightest. A computer that is actually fluent in English — as in, understands the language and can use it context-appropriately — should blow your entire mind.

> A computer that is actually fluent in English — as in, understands the language and can use it context-appropriately.

Did you never do grammar diagrams in grade school? :-)

The "context" and structure of language is a formula. When you have billions of inputs to that formula, it's not surprising you can get a fit or push that fit backwards to generate a data set.

This algorithm does not "understand" the things it's saying. If it did, that wouldn't be the end of the chain. It could, without training, make investment decisions on that advice, because it would understand the context of what it had just come up with. Plenty of other examples abound.

Humans or animals don't get to have their firmware "upgraded" or software "retrained" every time a new hype paper comes out. They have to use a very limited and basically fixed set of inputs + their own personal history for the rest of their lives. And the outputs they create become internalized and used as inputs to other tasks.

We could make 1M models that do little tasks very well, but unless they can be combined in such a way that the models cooperate and have agency over time, this is just a math problem. And I do say "just" in a derogatory way here. Most of this stuff could have been done by the scientific community decades ago if they had the hardware and quantity of ad clicks/emails/events/gifs to do what are basically sophisticated linear algebra tasks.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#159

Post by exolymph »

nullc wrote: post #132
Filligree wrote: post #77

Earlier quoted context omitted.

If you want to play with GPT-3, you can do so right now. Go to https://play.aidungeon.com Make an account, and select the "Dragon" model. That's GPT-3. I've spent ten hours playing with it over the last two days. It isn't perfect, and it feels short of the hype it's generating about itself, but it's an amazing leap nonetheless. It really seems to have an understanding of causality, biology, all sorts of fictional the…

The domain is play.aidungeon.io and the GPT3 based version is only available to sponsors right now. After seeing that the domain name didn't work I thought for a moment that your post was GPT3 output-- imaginary URLs is a good GPT2 tell--, but some research shows that there actually is a GPT3 version: https://medium.com/@aidungeon/ai-dungeon-dragon-model-upgrad...

It's only $10 to get access.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#160

Post by qotgalaxy »

wcoenen wrote: post #146
m3kw9 wrote: post #106

Earlier quoted context omitted.

Problem was I lost interest half way because it lost my interest after the 2nd paragraph. For those that say it was good till the last few is really pretending to understand what it said. It really did not made much sense.

It's ironic that you're critiquing the algorithm for not making sense, while contradicting yourself in your first sentence. Did you lose interest "half way", or "after the second paragraph"? It can't be both.

I'm assuming that the comment that you are replying to was generated by GPT-3. Maybe this one too?
Post reply on HN