OpenAI's GPT-3 may be the biggest thing since Bitcoin • Hacker News Board

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#261

Post by ilaksh » Sun, Jul 19, 2020, 5:33 AM UTC

Tarrosion wrote: post #120

sinity wrote: post #78

Earlier quoted context omitted.

Uh-huh. Except anyone can verify the results themselves, sans specific ones like code generation - through (paid version) of AI Dungeon. Amount of people confidently posting bullshit on Hacker News is astounding. Reacting like everything we know about GPT is just a bunch of tech demos. Supposedly everyone who has access to the API is just a shill. Eh.

I just spent an hour or so playing around with the paid version of AI Dungeon, and was super unimpressed. It's pretty fun for a moment, sure, and I assume some really heroic work went into building it. I'm not saying the creators did a poor job so much as the task is really hard and the final result is...lacking. The "Dragon" (GPT-3) engine responds reasonably to any particular input, but clearly lacks a coherent sta…

Right. It has some type of language ability, but no world modeling. So overall it really doesn't make sense.

But since that is so obvious, I assume many people are trying to figure out how to improve it. So I am excited to see if they can make progress in the next few years.

It is going to be quite difficult though. I think it might require integrating a totally different type of subsystem, if it is possible at all.

But the ability to make realistic sounding language is a step forward it seems to me.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#262

Post by zerocrates » Sun, Jul 19, 2020, 5:34 AM UTC

tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was.

I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of scale?

A widespread ability to pretty convincingly fake more difficult things like photos and video seems much more relevant or "disruptive" than anything in the realm of text generation. I just don't really know what "fake" text does at the end of the day.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#263

Post by lopmotr » Sun, Jul 19, 2020, 5:36 AM UTC

abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

No new technology is impressive when it comes incrementally. A camera that automatically records the latitude and longitude of where each photo was taken would have blown my mind as a child. I couldn't have conceived any way it might have worked. But nearly all cameras do that now and at most it's a curiosity or a privacy worry, not a blown mind.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#264

Post by KKKKkkkk1 » Sun, Jul 19, 2020, 5:37 AM UTC

Some posters here are expressing alarm at how good this example of GPT-3 generated writing is. I wouldn't be so alarmed. The Internet is already filled with quasi-coherent misinformed writing that's generated by humans in good faith. That's exactly the reason why we rely on our peers' judgement through sites like HN or through social media to select the bits that deserve our attention despite being misinformed and quasi-coherent.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#265

Post by richk449 » Sun, Jul 19, 2020, 5:39 AM UTC

nullc wrote: post #157

Dangeranger wrote: post #131

Earlier quoted context omitted.

Can you elaborate on how you are having GPT2 contribute to your comments? What is your process?

I write a prompt, often copying text from the articles or other comments, and have it generate a lot of completions. I skim over the completions and grab interesting parts. For example, complaining about "quoted text" in my above comment was GPT2's suggestion (and also an actual issue with GPT2 which it was exhibiting by producing that text). Actually, all the text above from "There are a few" and beyond were written…

Pretty much everyone agrees that these models, no matter how good they are at writing, don’t understand what they are writing, and can’t develop new ideas, only combine existing ideas together.

Given that, it’s seems that what you are doing adds only noise, and no value, to HN.

Of course, that is probably also true of a large fraction of pure-human posters ...

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#266

Post by ajzinsbwbs » Sun, Jul 19, 2020, 5:39 AM UTC

quotemstr wrote: post #150

oehtXRwMkIs wrote: post #85

Earlier quoted context omitted.

For most of human history text has not existed. I think you are conflating language with written language which is quite common in the post-Gutenberg age. I think sentience like most things is a spectrum, so I'm not really sure what you mean by sentient, but I would argue that for most people the bar for sentience is much higher than text prediction. The Chinese room is only one aspect of our minds, and we don't even…

I have to disagree with this perspective and reiterate my original position: language is sentience. And to be fair, reasonable people stake out positions on both sides of this debate: I'm not claiming that the alternative proposition is somehow unreasonable. It's a legitimate subject of scholarly disagreement. Nevertheless, I'm still firm on language. Why? Because all complexity is ultimately about symbolic manipulat…

I think you only need language to communicate, it’s not necessary in order to think. Do you agree that a human that grows up in isolation probably won’t develop a language? Would you say such a human isn’t sentient?

I think as highly social beings we often annotate all of our thoughts with the language we could use to communicate them, which could lead us to believe that the thoughts are indistinguishable from the language, but that conclusion seems like an error to me. I’ve also heard some people talk about how they are “visual” or “geometric” thinkers and sometimes think in terms of images and structures without words.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#267

Post by kortilla » Sun, Jul 19, 2020, 5:40 AM UTC

ipnon wrote: post #195

tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

We must return, regretably, to judging the quality of a text on the content within itself, instead of by appeal to the authority who wrote it. This will all change if we can agree GPT-{3,4,...,n} is the greatest author on this planet. To be honest I find it hard to believe that won't be the case within this century.

Right, when judging an individual piece of text you should evaluate the merits of the argument. Where this becomes dangerous though is in building what looks like consensus among thousands of users of a service.

Text like this makes it feasible to build huge puppet networks that can get upvotes/karma from real users quite easy by playing into their echo chambers.

This type of technology could easily be the undoing of anonymous upvote driven forums like reddit/hn.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#268

Post by zerocrates » Sun, Jul 19, 2020, 5:41 AM UTC

postingawayonhn wrote: post #176

Keyframe wrote: post #12

disruptive potential comparable to that of blockchain technology. When I read this, I don't know, genuinely, if it's a _potential_ blockchain has/had or a realized potential, which I find hard to believe since it didn't disrupt anything apart gamblers so far.

The bot wrote the article and presumably it learnt that blockchain is disruptive from BitcoinTalk.

No, the human "author" supplied the "disruptive potential of blockchain" part.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#269

Post by sidcool » Sun, Jul 19, 2020, 5:41 AM UTC

ganfortran wrote: post #260

Bitcoin isn't big at all. It haven't made relevant changes to people's everyday life.

Do read till the end. I can understand the urge to comment. But believe me, it's worth it

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#270

Post by atdt » Sun, Jul 19, 2020, 5:41 AM UTC

The volume of machine-generated spam — microtargeted clickbait, specifically — on the internet could grow by an order of magnitude or more, making authentic human text a minority. That will hasten the demise of the internet as a democratic medium, because intelligent people will be increasingly reluctant to wager their time and attention on unvetted prose of uncertain provenance. This has been the trend for some years now, but it is going to accelerate very rapidly.