Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

471–480 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#471

Post by chrisjarvis »

ta_9c6abeb5 wrote: post #467
simonsarris wrote: post #80

I could tell this was GPT-3 because it reads like SEO-style sentence structure :/ > I imagine that similar results can be obtained by republishing GPT-3’s outputs to other message boards, blogs, and social media. I actually wrote a bit about this scenario and how it could explode comment sections to stonewall topics, which I'm calling Commentdämmerung: https://simonsarris.com/commentdammerung Still, I think a lot of…

What a beautiful thing is your website. I haven't felt this joy opening a page in ages.

I was thinking the same thing until those animated dots continued to fly around the screen while I was trying to read the words. A strange design choice...

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#472

Post by quotemstr »

someguyorother wrote: post #375
quotemstr wrote: post #25

Earlier quoted context omitted.

> AI text-generation "Text generation" undersells it a little bit. What are humans except "text generation" machines? Language is the stuff of reason. GPT-3 has demonstrated capabilities that we believed were exclusive to humanity --- humor, logic, sarcasm, cultural references --- in an automatic and generalizable way. It's far more than a "text generation" system. I look forward to seeing what GPT-4 and GPT-5 can do…

>What are humans except "text generation" machines? Text generation doesn't chop wood, optimize speedruns, build machinery or win 100-metre dashes. Text may be involved in training for these things, but to say that doing them is text generation would be like saying that... since compiling code and running AlphaZero both generates bits, AlphaZero is a compiler.

The ability to do these tasks is neither necessary nor sufficient for recognizing something as human. Helen Keller was human after all. What differentiates us is language.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#473

Post by nschucher »

polyanos wrote: post #423
killerstorm wrote: post #378

Earlier quoted context omitted.

Why do you believe they won't release the model? It's Open AI after all. Google, NVIDIA and Microsoft also make their models freely available. Google already trained a model which is bigger than GPT-3. There might be some delay, but no fundamental problem with it. Fine-tuning the entire GPT-3 is impossible on a single GPU. But it's still possible to fine-tune specific layers. Plus, I'm sure somebody will release a di…

>Why do you believe they won't release the model? It's Open AI after all. Because they want to commercialize the model "to cover the costs of research", it's stated in their blog post FAQ[0]. >Google already trained a model which is bigger than GPT-3. Source?, a fast Google search yielded no results and I'm curious. [0] https://openai.com/blog/openai-api/

They trained a >600B parameter translation model (GPT-3 is 175B parameters).

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (https://arxiv.org/abs/2006.16668)

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#474

Post by quotemstr »

patagurbon wrote: post #74
quotemstr wrote: post #50

Earlier quoted context omitted.

And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.

I don't believe we have any evidence this edition of GPT is capable of reasoning. I haven't experimented with it but I doubt it will respond correctly to even simple logic puzzles provided they are framed in a novel way (it may have already seen some puzzles, but I doubt it can extend that knowledge to a puzzle with different words)

This version of GPT can add, subtract, multiply, and divide without ever having been taught to do these things things. Yes, it can reason.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#475

Post by jimkleiber »

thewarrior wrote: post #164
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

I'm starting to sense that, in most scenarios, I will no longer want to engage in text-based conversations unless I know that I'm talking with a human. I already don't like spending a lot of time arguing with a bot on Twitter, this just makes it much more likely I'll also argue with a bot on medium-length text areas (e.g., HN, FB, Whatsapp, SMS, etc.) and maybe even on long-length text areas (e.g., Medium, blogs, NYTimes or things pretending to be real newspapers, etc.)

Second, I'm curious/terrified at how future iterations of GPT-3 may impact our ability to express ourselves and form bonds with other humans. First it's text messages and comments. Then it's essays. Then it's speeches. Then it's love letters. Then it's pitches. Then it's books. Then it's movie scripts. Then it's...

TLDR; Fascinated by the technology behind making something like this work and quite worried about the implications of the technology.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#476

Post by yaj54 »

cs702 wrote: post #147
thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

The thing that kills me is that soon enough, a "fake" paper written by GPT-3 will get published in an academic journal because it has actually contributed a new insight.

It's easy to consider text generation models as "just mimicking grammar". But isn't grammar also just a model of human cognition?

Is GPT modeling grammar or is it modeling human cognition? Since GPT can ingest radically more text (aka ideas) won't it soon be able to generate texts (aka ideas) that are a more accurate collation of current knowledge than any individual human could generate?

--

[Was this comment written by GPT-3?]

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#477

Post by nstart »

rasz wrote: post #414
nstart wrote: post #226

I'll be honest, I've followed the evolution of gpt's text generation and this is the first one that has successfully avoided the obvious contradictory nonsensical sentences. Social media also trains us to consume quickly rather than thoughtfully. Not a great look overall. As a whole, the article reads like someone who isn't very clear in what they want to say. Paragraph wise and section wise, the bot has generated a…

http://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.ht... Q: Which colorless green ideas sleep furiously? A: Ideas that are colorless, green, and sleep furiously are the ideas of a sleep furiously. Q: Who was president of the United States in 1600? A: Queen Elizabeth I was president of the United States in 1600. Q: How many Super Bowls do the Cincinnati Bengals win in the 2030s? A: The Cincinnati Bengals win tw…

I think I see what you are getting at here, but I feel like the two are different examples. Being able to generate blog posts that are vague but do not venture into an uncanney valley does not mean the same thing as passing the Turing test.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#478

Post by rainbow-road »

abernard1 wrote: post #196
abernard1 wrote: post #163

Earlier quoted context omitted.

> Hasn't the typical human taken in orders of magnitude more data than this example? Have you read GPT-3s 175 billion parameters (words, sentences, papers, I don't care) of anything? Do you know all the words used in that corpus? Nobody has or does. A child of a small age can listen to a very small set of things and not just come up with words to communicate with mama and papa what they learned, but they can reuse it…

> This post was generated using GPT-3. [;)] Your post was generated using GPT-3 and 175 billion parameters of pre-existing human writing, contextualized, distilled, and cross-referenced with terminology we've agreed on for centuries. It's a parrot, and I remain unimpressed. Take the learned knowledge of GPT-3 (because it must be so smart right?) and have it actually do something. Buy stocks, make chemical formulas, b…

> It's a parrot, and I remain unimpressed.

What's unimpressive about a stunningly believable parrot? I think, at the very least, that GPT-3 is knowledgeable enough to answer any trivia you throw at it, and creative enough to write original poetry that a college student could have plausibly written.

Not everything worth doing is as high-stakes as buying stocks, making chemical formulas, or building planes.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#479

Post by varjag »

kevmo314 wrote: post #204
abernard1 wrote: post #196

Earlier quoted context omitted.

> This post was generated using GPT-3. [;)] Your post was generated using GPT-3 and 175 billion parameters of pre-existing human writing, contextualized, distilled, and cross-referenced with terminology we've agreed on for centuries. It's a parrot, and I remain unimpressed. Take the learned knowledge of GPT-3 (because it must be so smart right?) and have it actually do something. Buy stocks, make chemical formulas, b…

> That post was generated using GPT-3 and 175 billion parameters of pre-existing human writing, contextualized, distilled, and cross-referenced with terminology we've agreed on for centuries. So basically like DNA?

Since literacy or human knowledge isn't encoded in DNA, it's nothing like it.
Post reply on HN