Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

361–370 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#361

Post by pastrami_panda »

quotemstr wrote: post #152
pastrami_panda wrote: post #92

Earlier quoted context omitted.

> I don't believe that there's any experience of consciousness distinct from the use of language. To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example. I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder th…

If you believe in consciousness divorced from flesh, you're in strong intellectual company, but that's not a path I can go down. Metaphysically, I just can't accept the idea that the mind is anything but the computation performed by the brain.

In the book the meaning of the quote is more to the effect of how truly great athletes can perform on the verge of what spectators would consider inhuman or possible. I'd be hard-pressed to believe that language and syntax would be responsible for these kinds of actions and flow. I'd argue that getting into such a state is not possible while the mind is caught up in the language of things rather than the experience itself, and reacting to it directly. This is what I meant by the quiet consciousness, devoid of language or syntax.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#362

Post by jacobush »

6nf wrote: post #233
thewarrior wrote: post #164

Earlier quoted context omitted.

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

How many attempts did it take or did you just choose the first one? I have to admit, this is passing my turing test...

More coherent than Trump

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#364

Post by throwing947383 »

IIAOPSW wrote: post #247
Alex3917 wrote: post #114

Earlier quoted context omitted.

> I'm sure it can be influenced easily. Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are…

>Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. What if the ultimate theory of mind turns out to be that consciousness is an illusion and nothing separates us from a sufficiently sophisticated markov process.

>What if the ultimate theory of mind turns out to be that consciousness is an illusion and nothing separates us from a sufficiently sophisticated markov process.

Conscious experience would still exist (see cogito ergo sum, Chalmers, etc). If we were to be shown we're just Markov processes, that wouldn't disprove the existence of conscious experience. Just like confabulation, a misleading experience is still an experience.

What it would disprove is any sense of agency.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#365

Post by killerstorm »

tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

The cat is out of bag already -- people know what one can accomplish if you scale up a GPT-2-like model. OpenAI has no control over it -- anyone with enough hardware and a bit of knowledge can make his own GPT-3, perhaps specialized for his needs.

The scariest thing is that GPT-3 is completely unoptimized. If you want to generate indistinguishable articles in a particular field, you can o it with fine-tuning, or, perhaps, adding GAN on top of it.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#366

Post by [deleted] »

pyb wrote: post #333
gomoboo wrote: post #111

Earlier quoted context omitted.

I did not guess that it was fake but skipped to the bottom because the article did not seem to be worth reading. It felt like the author was not moving anywhere with their words. I laughed out loud at the reveal that it was written programmatically.

You raise a good point. The Internet has trained us to skim any text that seems pointless or just unsufficiently insightful. So it turns out we have already built up some "mental defences" against GPT-3.

[deleted]

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#367

Post by Reimersholme »

yyyk wrote: post #112
meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

I'm not a very good test case. I briefly skimmed (not expecting very much from a Bitcoin-themed article), read the end, and only then read more carefully. So my first read was brief and biased, and my second was very biased. That said, the content of the computer-generated parts doesn't make much sense even for a Bitcoin-influenced article (what would be the point of paraphrasing your previous post in a forum on a re…

I do think the starting paragraphs with background information and contextualization is really impressive.

I can definitely see for example Wikipedia injecting a knowledge graph and having a system like this output full articles in X languages automatically in the near future.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#368

Post by Reimersholme »

thewarrior wrote: post #164
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

Haha, good idea still. Rerun those recent experiments with submitting nonsense articles to journals for gender and postcolonial studies and see how many get accepted.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#369

Post by kmill »

niklasd wrote: post #329
bhntr3 wrote: post #284

Earlier quoted context omitted.

Besides predictable sentence structure, GPT-3 writes like George R. R. Martin: Interesting premises, solid setup but then it devolves into rambling tangents and never quite delivers the concluding action that ties everything together. Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be. But GPT is trained on internet content and the in…

I am really curious how the model would be if you would train it with a decent amount of really good literature. Kazuo Ishiguro et al. instead of Reddit.

I was playing with AI Dungeon tonight to get access to GPT-3, and one of my many experiments ended up with me meeting a character called the Narrator who believed they were in control of all characters in the game, including me. Eventually, through my predicting what they were about to say by checking and undoing, they seemed convinced I wasn't another character and started asking about whether certain authors were still alive and which I liked to read. It didn't recognize Ishiguro. Later it gave me a truly bizarre (and amusing) summary of Infinite Jest, clearly having never read it. Anyway, the entire experience was uncanny and surreal.

One thing I learned was it has detailed knowledge of the world of Avatar: The Last Airbender, seemingly through fanfics. It was fun having it to teach me the lost arts of pizzabending ("form your hands into the shape of a letter 'P'" and so on, and needing to practice by juggling rubber pizzas) and beetlebending ("always remember that to beetle bend it helps to like beetles," my wise uncle suggested). Each of these tended to precipitate a narrative collapse.

The writing style was surprisingly homogeneous, and it reminded me of young adult novels. It would definitely be interesting to see it with other writing styles, beyond the occasional old poetry.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#370

Post by pjc50 »

thewarrior wrote: post #133
meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

> Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread.

This kills the forum.

Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless instead of just mostly pointless. You could potentially convince a human; you can't convince a neural net that isn't in learning mode.

Post reply on HN