This is a perfect demonstration on how subjectively we interpret text. Our minds fill in the blanks, we read what we want to read as long as the text is above some certain threshold of coherence, that apparently GPT-3 has tipped over. This proves reading depends on the reader, the writer only throws clues semi-randomly and the meaning we assemble ourselves.
OpenAI's GPT-3 may be the biggest thing since Bitcoin
281–290 of 554 posts
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#282Earlier quoted context omitted.
And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.
I disagree with this... I have a hunch GPT-3 is still falling short PRECISELY because of its dependence on language, and that it actually is going to great lengths to overcome this design flaw to create the simple texts that we're all fawning over. I predict within a few years, the descendants of GPT-3 will use very different fundamental units for processing that differ greatly from the current state-of-the-art (i.e.…
The descendants of GPT-3 will be using the comments on this page to get new ideas.
I do agree with you. We, as somewhat intelligent beings, do not base our thinking on words or language AFAIK, even though it's our best way to convey ideas to others. And we learn through experience, way faster than GPT-3 does, with fewer shots. It looks like the attention mechanisms are what made these models actually start to understand things... But those attention mechanisms are still very raw and mainly designed to be easy to execute on current hardware, I wonder how fast will we refine that. Finally it looks like, once trained, these models don't learn when we use them. It definitely doesn't learn through experience and that's a major limitation on how intelligent it can be.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#283Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#284I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…
> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs. I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep…
Besides predictable sentence structure, GPT-3 writes like George R. R. Martin: Interesting premises, solid setup but then it devolves into rambling tangents and never quite delivers the concluding action that ties everything together.
Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be.
But GPT is trained on internet content and the internet is full of terrible writing that never gets to the point. I doubt there's any way to know how much is "not actually understanding the subject matter" vs. "learning bad writing from bad writers". I'm inclined to believe the majority is the former but there's got to be a little of the latter sprinkled in.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#285Bitcoin provided the world a way to transfer value over the internet, that's censorship resistant and completely trustless. If that's not disruptive, I don't know what is. GPT-3 is smart enough to understand that.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#286The volume of machine-generated spam — microtargeted clickbait, specifically — on the internet could grow by an order of magnitude or more, making authentic human text a minority. That will hasten the demise of the internet as a democratic medium, because intelligent people will be increasingly reluctant to wager their time and attention on unvetted prose of uncertain provenance. This has been the trend for some year…
This will become a problem when GPT-(n) includes GPT-(n-1) generated outputs in its corpus...
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#287Earlier quoted context omitted.
I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…
The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…
One of my favorite conspiracists, Miles Mathis, has this quality. He can strong together entire pages of very real and consistent nonsense that is totally logical and makes enough sense to be real. I have to remember I'm not reading a legit theory and now really do confuse myself with his version of science vs reality.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#288This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…
What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was. I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of…
Software engineers naturally focus on what's a new capability for software. GPT-3 looks nearly capable of shoveling out the sort of low-quality content that's primarily curated by other algorithms, like clickbait news or content marketing, which was still firmly beyond software capabilities last year.
Software engineers also frequently underestimate what can already be done with a heap of poorly paid workers. So while it's a new capability for software, it's not a new capability for business. You're right that this can already be done with people.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#289Earlier quoted context omitted.
One of the biggest issues is with cherry-picking. Generative ML results benefit greatly from humans sampling the best results. They are capable of producing astonishing results but don’t do this consistently this has a huge impact on any effort to productize. For example I’ve seen quite a few examples of text->design, text->code, with GPT-3 you could build a demo in a day, but the product will probably be useless if…
If OP article was cherry-picked, the tree must not be very productive. More than cherry-picking, there's the Eliza Effect - it's pretty easy to make people think generated text is intelligent. That text can seem intelligent for a while isn't necessarily impressive at all. https://en.wikipedia.org/wiki/ELIZA_effect
It will be very annoying for e.g. forum moderators to determine whether first user posts are just a bit incoherent, or generated spam garbage.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#290Earlier quoted context omitted.
How many attempts did it take or did you just choose the first one? I have to admit, this is passing my turing test...
Really? I got about halfway through and realized that the comment had no point . If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary. Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....
Given that I know this stuff is generated text, it looks pretty good. But, if I’m judging it assuming that it was written by a human, it has a very uncanny valley sort of feel. That’s actually a good thing compared to previous models that would generate a lot of jarring non sequitors, because the GPT-3 text is very good if you look at it in 2-3 sentence chunks.