Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

291–300 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#291

Post by exolymph »

abernard1 wrote: post #158
exolymph wrote: post #149

Earlier quoted context omitted.

> This does not impress me in the slightest. A computer that is actually fluent in English — as in, understands the language and can use it context-appropriately — should blow your entire mind.

> A computer that is actually fluent in English — as in, understands the language and can use it context-appropriately. Did you never do grammar diagrams in grade school? :-) The "context" and structure of language is a formula. When you have billions of inputs to that formula, it's not surprising you can get a fit or push that fit backwards to generate a data set. This algorithm does not "understand" the things it's…

I bet you can't guess which parts of this are me versus the AI: https://pastebin.com/FHiRR95F

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#292

Post by vidarh »

geofft wrote: post #238
6nf wrote: post #233

Earlier quoted context omitted.

How many attempts did it take or did you just choose the first one? I have to admit, this is passing my turing test...

Really? I got about halfway through and realized that the comment had no point . If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary. Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....

It felt like it was making a slightly ranty observation that scientists are already trying to much to be philosophers than to actually do science that changes the world, yet science has brought us far enough that it acts as an enabler for all kinds of pop-philosphers.

The final bit doesn't quite connect, but overall I've seen far less coherent comments written by humans on subject with far more logical flaws.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#293

Post by t_von_doom »

peterthehacker wrote: post #220
colordrops wrote: post #154

Earlier quoted context omitted.

> I'll be impressed the day I can see a program that can 1) only rely on its own limited experiential inputs Hasn't the typical human taken in orders of magnitude more data than this example? And the data has been of both direct sensory experience and texts from other people as well.

I think that it’s the opposite. This algorithm requires many examples of text on the specific topic. Probably more than most humans would require. > While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples [0] I don’t know what constitutes an example in this case but let’s assume it means 1 blog article. I don’t know man…

> assume it means 1 blog article

I'd like to play devils advocate here.

Given one blog article in a foreign language: Would a human be able to write coherent future articles?

With no teacher or context whatsoever how many articles would one have to read before they could write something that would 'fool' a native speaker? 1000, 100,000?

I have no idea how to measure the quantity/quality of contextual and sensory data we are constantly processing from just existing in the real world, however, it is vital to solving these tasks in a human way - yet it is a dataset that no machine has access to

I would argue comparing 'like for like' disregards the rich data we swim amongst as humans, making it an unfair comparison

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#295

Post by seek3r00 »

“I could not stop thinking about the applications of such a technology and how it could improve our lives.

I was thinking of how cool it would be to build a Twitter-like service where the only posts are GPT-3 outputs.”

This could have been either the output of GPT-3 or someone who doesn’t know what they’re saying.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#296

Post by mr__y »

yyyk wrote: post #93
threeseed wrote: post #79

Earlier quoted context omitted.

The truly scary part is SEO where GPT-3 could ruin search engines overnight. Google at this point favours long form content for many search intents. Being able to generate thousands of these pages in one-click is a real problem. Not just because of popular topics e.g. "covid-19 symptoms" but more so for the long tail e.g. "should I drink coffee to cure covid-19".

Quite a lot of SEO already uses simple word generation techniques. It isn't clear GPT-3 is an improvement there - human text recognition might not be whatever Google does. It may be that Google's algorithms don't care at all how human-like the text is, or that their own recognition algorithm/NN (whatever they use) isn't fooled. Even if it is affected, Google has the money and corpus to build its own competing NN to r…

While I have no doubts that they could build NN capable of recognizing GPT-3 text I believe that this would still pose a problem given the amount of content to be analyzed at the scale that Google deals with

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#297

Post by metrokoi »

abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

The article says more about the state of tech blogging than it does GPT-3. I kept thinking "great, another one of these, when are they actually going to show me any results?"

We've been conditioned to accept articles where there's a lot of words and paragraphs and paragraphs of buildup, but nothing actually being said.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#298

Post by johndough »

httpsterio wrote: post #141

After reading the AI generated sections, I have to say that I'm mostly quite impressed. As I lack the context as I am not actively following the ML and procedurally generated text scenes for years, it was still just markov chains back then, I can't say for sure how accurate the produced text was. It's scary though. Many commenters are only discussion about the business opportunities and path to profitability, but if…

The power that GPT-3 gives to spammers has me worried as well. Anyone with sufficiently many IP addresses can now single-handedly kill any forum by spamming it with text which is at first sight indistinguishable from the average poster.

Automatically generated spam could also be used to suppress discussion of political opinions or certain topics by drowning them in a sea of garbage comments, which seems highly problematic.

We should come up with a solution to combat this. Here are a few ideas:

- To a certain extend, the comment voting mechanism can be used to filter out comments which lead nowhere. If that stops working because comments are too good, that is not a problem. In the words of Randall Munroe: https://xkcd.com/810/ However, voting only works if the number of people voting on a certain comment outnumber the spam comments, so this will fail with too many spam comments.

- Another solution would be to verify that every comment author is an actual human being. The GPG web-of-trust could be used for that, but this is untraceable for the average user. There also should be an additional layer of indirection between actual user identities and online identities to preserve privacy, but I am not sure if it is possible to have both privacy and limit a forum user to a single account so they can not circumvent a ban by simply creating new accounts. A trusted third party could solve this, but a distributed solution would of course be preferable. Maybe there is a smart cryptographic solution to this problem?

- For the near future, every provider of GPT-3-based services should also provide a service to check whether some given comment has been generated by their model. This can be realized by hashing substrings of all generated output and storing the hashes in a bloom filter. This is not a long term solution since technological advancement will soonishly enable regular people to train similar models.

- Training other neural networks to detect automatically generated text is not a solution because the generating networks can simply be trained to not be detected by the detecting networks. This is just a cat and mouse game.

Does anyone have a better solution?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#299

Post by atq2119 »

Trasmatta wrote: post #139
tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

> Now we can't trust text, the most trusted medium in human history, and then what? I don't understand what you mean by this. Isn't text the least trusted medium? Anyone can easily try to impersonate anyone else in text. Going by your argument of trust, shouldn't we be more afraid of deepfake videos? Or even more accurately, deepfake videos paired with something like GPT-3.

Yes, we should also be afraid of deepfake.

With GPT-3, the issue isn't so much being able to impersonate somebody, but the ability to generate human-seeming text at scale. This allows you to create a false perception of public sentiment by maintaining fake accounts on online forums, writing fake letters to the editor, to politicians, and so on.

On the flip side, this is already possible today with content farms, and perhaps GPT-3 can save us by being the thing that finally erodes our trust in these signals.

One thing is sure though, it drives yet another arms race, and arms races are always a net loss for society.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#300

Post by samvher »

zerocrates wrote: post #262
tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was. I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of…

I rely a lot on text for obtaining information and shaping my opinion, and in many cases short form text plays an important role (e.g. here or on reddit). I’m sure I’m not alone in that.

This technology can at the very least waste my time, confuse me and hide the content that I’m actually looking for. It looks like it can feasibly generate 2-3 sentence comments that make sense in context, but in an automated way, with the purpose of injecting a specific sentiment into a comment section.

I already didn’t like that sometimes it seems comments I think are written by humans might not be (or they might not be sincere). This kind of technology can make that problem a lot larger.

It could flood the internet with so much crap, that is so hard to filter out, that the internet becomes a much less usable source for obtaining reliable information. I think that’s pretty scary.

Post reply on HN