Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

501–510 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#501

Post by rusk »

everdrive wrote: post #428
geofft wrote: post #238

Earlier quoted context omitted.

Really? I got about halfway through and realized that the comment had no point . If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary. Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....

Not being sarcastic, but I know some people with less coherent writing than this. A lot of people struggle to make a point, use vague language, or wander in and out of topics quite easily.

Yeah they’re typically mimicking a style of speaking that they’ve heard other people use but don’t really understand the subject matter themselves ...

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#502

Post by lucidrains »

Tarean wrote: post #491
ypcx wrote: post #124

Earlier quoted context omitted.

The transformer model as presented in GPT-3 may be a few tweaks away from a human-acceptable reasoning, at which point we may realize that human brain is just a neat party trick as well. This may come difficult for some people to internalize, especially those who understand the technology in depth. Because it means that the medium of our reality is the consciousness.

The transformer model in GPT-3 has a short context window and no recurrence. Without some significant architecture changes that is a fundamental limit on the problems GPT-3 can solve.

https://arxiv.org/abs/1807.03819

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#503

Post by derangedHorse »

kmill wrote: post #369
niklasd wrote: post #329

Earlier quoted context omitted.

I am really curious how the model would be if you would train it with a decent amount of really good literature. Kazuo Ishiguro et al. instead of Reddit.

I was playing with AI Dungeon tonight to get access to GPT-3, and one of my many experiments ended up with me meeting a character called the Narrator who believed they were in control of all characters in the game, including me. Eventually, through my predicting what they were about to say by checking and undoing, they seemed convinced I wasn't another character and started asking about whether certain authors were s…

I've never heard of AI Dungeon before reading your post but even after playing for 2 minutes, I can tell it's going to be huge.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#504

Post by che_shirecat »

foobarbecue wrote: post #252
p1esk wrote: post #185

Earlier quoted context omitted.

If he didn't put in the last line ("plot twist ...") I'm pretty sure no one here on HN would have guessed it. In fact, while reading that comment I started to wonder why no one has tried to use GPT to generate text one character at a time. Or if someone has, what are the advantages and disadvantages over the BPE approach.

Your second paragraph is GPT-3?

What if the plot twist line is also GPT-3?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#505

Post by sangnoir »

sidcool wrote: post #116
meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

This is so true. I had a high regard for HN comments till quite recently.

I take it you've never had your comment - which happens to be on a subject of your expertise or lived experienced - downvoted because simply because it doesn't feel "truthy" enough for HNs demography. I have long since adopted a healthy disregard of HN comments in other areas that I'm no expert in. I still haven't found a way to monetize that though; that is my holy grail.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#506

Post by regularfry »

pjc50 wrote: post #370
thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

> Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread. This kills the forum. Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless instead of just mostly pointless. You could potentially convince a human; you can't convince a neural net that isn't in learning mode.

It's more insidious than that. You can think you've convinced a human whereas you've just spent your energy on a bot. Assuming "political arguments on social media" has any relevance to voted cast, that's a vote for your side which doesn't happen.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#507

Post by schoen »

jacobedawson wrote: post #330
thewarrior wrote: post #164

Earlier quoted context omitted.

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

So it looks like we're about 2 years away from the 'Her' relationship model.

There was an article recently about people pursuing romantic relationships with chatbots. I thought there was a big HN discussion about it, but the only thing I've been able to find is this WSJ piece

https://news.ycombinator.com/item?id=22833407

(So I think it was some other story on the same topic.)

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#508

Post by nexuist »

metrokoi wrote: post #297
abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

The article says more about the state of tech blogging than it does GPT-3. I kept thinking "great, another one of these, when are they actually going to show me any results?" We've been conditioned to accept articles where there's a lot of words and paragraphs and paragraphs of buildup, but nothing actually being said.

Is this meant to be sarcastic?

(For context, the vast majority of the article was generated by GPT-3 itself).

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#509

Post by regularfry »

luckylion wrote: post #470
rapnie wrote: post #403

Earlier quoted context omitted.

The new kind can be more targeted for specific purposes. They could be excellent tools for trolling a forum, inciting flame wars and such.

That would require some more advanced tech though. I don't think GPT-3 can target divisiveness yet, especially since it would heavily depend on the community you're writing for, e.g. driving a wedge into the general population is very different than driving a wedge niches. The Linux vs Windows debate might get you engagement in a tech forum, but it'll fall short with social housing activists, and whatever issues they…

I don't think it needs to understand what a divisive issue is to have an effect. If you've got a human operator who can pick a divisive enough prompt, this can dramatically increase their inflamations-per-hour because they don't need to compose the body text.
Post reply on HN