Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

491–500 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#491

Post by Tarean »

ypcx wrote: post #124
jcahill wrote: post #53

Earlier quoted context omitted.

GPT-3 is a neat party trick. But the things that'll be done with web archives* in the next 20y will make it look like the PDP-8. ~love, a web archivist * GPT-3 is trained on one

The transformer model as presented in GPT-3 may be a few tweaks away from a human-acceptable reasoning, at which point we may realize that human brain is just a neat party trick as well. This may come difficult for some people to internalize, especially those who understand the technology in depth. Because it means that the medium of our reality is the consciousness.

The transformer model in GPT-3 has a short context window and no recurrence. Without some significant architecture changes that is a fundamental limit on the problems GPT-3 can solve.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#492

Post by 29athrowaway »

g_p wrote: post #372
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

There's actually precedent for this - sci-gen got papers accepted in IEEE and Springer publications, through peer review, and they had to investigate their back catalogue to look for others. Given the propensity for academic writing to often favour the strategy of confusing the author through obfuscation (to make a minor advance sound more significant than it is), I suspect tools like this could, as you say, actually…

Dr. Herbert Schlangemann (fake alias for Sci-gen) not only got papers accepted in journals, it was invited to participate as session chair at conference co-sponsored by IEEE.

https://en.wikipedia.org/wiki/SCIgen#Schlangemann

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#493

Post by avivo »

zerocrates wrote: post #262
tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was. I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of…

Here's a paper I co-authored about the broader implications: https://thoughtfultech.org/malicious-use-synthetic-media-mac...

There's a bunch more out now.

This might also be helpful: https://medium.com/@aviv/what-does-a-world-with-automated-so...

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#494

Post by quotemstr »

oehtXRwMkIs wrote: post #167
quotemstr wrote: post #150

Earlier quoted context omitted.

I have to disagree with this perspective and reiterate my original position: language is sentience. And to be fair, reasonable people stake out positions on both sides of this debate: I'm not claiming that the alternative proposition is somehow unreasonable. It's a legitimate subject of scholarly disagreement. Nevertheless, I'm still firm on language. Why? Because all complexity is ultimately about symbolic manipulat…

I think you're mixing up sentience/consciousness/intelligence. Many animals are sentient for example but as far as we know, they don't really have language. But I think I get what you're getting at, which I believe is "human level intelligence requires langauge". I think that's a reasonable take. But you said "sentience", and you said "is", which makes your position difficult to agree with.

Yes. You're right. I was sloppy with language. To be specific, I think that "human level intelligence" is basically synonymous with "able to think about thinking", and I think to do that, you need symbolic manipulation, and language is the only way we can do symbolic manipulation.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#495

Post by Naracion »

walleeee wrote: post #170
ypcx wrote: post #124

Earlier quoted context omitted.

The transformer model as presented in GPT-3 may be a few tweaks away from a human-acceptable reasoning, at which point we may realize that human brain is just a neat party trick as well. This may come difficult for some people to internalize, especially those who understand the technology in depth. Because it means that the medium of our reality is the consciousness.

Was this comment generated by GPT-3?

I doubted that as well, but I don't think it is--at least it's not a simple copy paste. There's an emphasis on _is_ in the last sentence which I don't think the algorithm could have generated.

However that makes one wonder if it can also learn to generate emphases, and if so, how would it format? With voice generation it can simply change its tonality but with text generation it has to demarcate it in some way--does the human say "format the output for html", for instance?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#496

Post by Udik »

yaj54 wrote: post #476
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

The thing that kills me is that soon enough, a "fake" paper written by GPT-3 will get published in an academic journal because it has actually contributed a new insight. It's easy to consider text generation models as "just mimicking grammar". But isn't grammar also just a model of human cognition? Is GPT modeling grammar or is it modeling human cognition? Since GPT can ingest radically more text (aka ideas) won't it…

My impression is that these models are already doing far more than what the language production machinery in our brain does. We are able to produce language according to grammar and semantics, but we also have independent mental representations to guide the generation of language and to provide context.

I don't really understand why we're trying so hard to build models that can generate coherent texts based on having predigested only other texts, without any other experience of reality. Their capabilities appear already superhuman in their ability to imitate styles and patterns of any kind (including code generation, images, etc.). It feels like we're overshooting our target by trying to solve an unsolvable problem, that of deriving the semantics of reality from pure text, without any other type of input.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#497

Post by ianhorn »

colordrops wrote: post #154
abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

> I'll be impressed the day I can see a program that can 1) only rely on its own limited experiential inputs Hasn't the typical human taken in orders of magnitude more data than this example? And the data has been of both direct sensory experience and texts from other people as well.

GPT-3 was trained on half a trillion words (common crawl, webtext, two book corpuses, and wikipedia, IIRC). At about 100 words per minute, that's almost ten thousand years of continuous speech. By my estimate it's probably a few thousand times what people actually hear in a lifetime. We don't experience nearly the volume of language that it did.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#498

Post by regularfry »

thewarrior wrote: post #164
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

In possibly an unwise move, I'm actually going to respond to the initial point here.

There's a totally valid discipline in taking concepts from different areas and smushing them together to make a new idea. That's what a lot of creativity is, fundamentally. So a bot that's been trained across a wide variety of texts, spitting out an amalgam in response to a prompt that causes a connection to be made, is not only possible, but likely a very good way of generating papers (or at least abstracts) for humans to check. And if the raw output is readable, why not publish it?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#500

Post by rusk »

EGreg wrote: post #450
ascar wrote: post #340

Earlier quoted context omitted.

Also both comments made me suspicious half way through and I scrolled to the bottom to check for a GPT-3 note. Without that note I would definitely have regarded it as incoherent rambling by a human. Especially the second comment can be coherently interpreted with some good will and a cynical view of the humanities and philosophy. The "author" could say that once GPT-3 can write humanities papers it will quickly make…

Who here thinks some Donald Trump's answers were written by an early version of GPT3, designed to produce more bombastic and rambling rhetoric than usual?

In principle it’s not too far fetched ... there’s almost certainly some kind of data-driven algorithmic processing going into a lot of speech writing these days; some of the drops are so implausible they’d almost certainly have to have been suggested by a machine!
Post reply on HN