Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Trains Language Model, Mass Hysteria Ensues

approximatelycorrect.com

Post reply on HN
Print view View original

81–90 of 119 posts

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#81

Post by gfodor »

kirillzubovsky wrote: post #49

What if OpenAI didn’t write the piece? What if the research was announced by the machine, and the folks at OpenAI are all dead?

You joke, but there's a real point here -- many commenters in this thread are complaining that OpenAI's position on this is a marketing stunt. Presumably, if this stuff gets commercialized, it will probably be adept at a few domains first, and I feel like writing good marketing copy will be one of them. So perhaps the bot itself didn't do so here, but it wouldn't surprise me if a self-marketed bot exists in the near future.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#82

Post by Scaevolus »

pas wrote: post #72
roenxi wrote: post #54

Earlier quoted context omitted.

If you look at it as a PR stunt, it is almost certainly a good idea. If a bad actor can auto-generate text that is not really distinguishable from something written by a human, how does a community with open membership (eg, HN) protect itself? I imagine this technology will enable interesting new attacks against online communities; we havn't seen that for a while. OpenAI are extremely sensible to draw attention to th…

But ... it's not novel. We could already generate convincing gibberish years ago. Now the novelty is that this can be better targeted. But even simple Markov-chain based text generators were good enough to fool people for a bit. And there was always people that had too much free time to write. A lot. (See for example the crackpots and conspiracy theorists that bombard physics forums. See the 9/11, Zeitgeists, etc. mo…

Markov-chain generators are extremely lacking in long-term coherency. They rarely even make complete sentences, much less stay on topic! They were not convincing at all-- and many of the GPT-2 samples are as "human-like" as average internet comments.

Conjecture: GPT-2 trained on reddit comments could pass a "comment turing test", where the average person couldn't distinguish whether a comment is bot or human with better than, say, 60% accuracy.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#83

Post by robomc »

amrrs wrote: post #8

>Elon Musk distances himself from OpenAI, group that built fake news AI tool This is the worst headlines in this matter. This is one of the leading media in India. A language model being touted as Fake news AI tool. This is like calling a car, A run over machine by Ford. https://www.hindustantimes.com/tech/elon-musk-distances-hims...

That's not quite fair. The sample output they're touting is really nothing other than false text (when it's coherent), almost all of which is in the style of news.

So for the Ford analogy to be apt, Ford would have to have designed a car nobody has ever seen, and released a video which is basically just hundreds of hours of the car running people over.

I mean, a car has lots of well understood non-running-people-over capabilities. But have they demonstrated that this model is useful for anything other than generating fake news-sounding spam text?

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#85

Post by [deleted] »

Cacti wrote: post #75
bilbo0s wrote: post #58

Earlier quoted context omitted.

> You are fooling yourself if you think your stance makes a single mote of difference in this arms race... In fairness, if that's true, then no one has any need of her model. More seriously speaking, why does anyone need, say, "training set x", or "model y", to make their implementation work? You don't. So I don't really understand why everyone is so worked up about not releasing this stuff? If you want to do it, do…

No one is saying "I demand everyone do it." There are two points: - If they are going to publish the research, and want to claim it as research (which they will, either by submitting it to a conference or putting on arxiv for the citations), then they should publish the supporting material, because without the supporting material it is impossible for reviewers or other researchers to evaluate. This is not just the mo…

[deleted]

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#86

Post by dogma1138 »

roenxi wrote: post #54
pishpash wrote: post #42

Earlier quoted context omitted.

Exactly. This is like holding up spam samples or how spammers operate from the spam detecting work. That side (and the cultural discussions) needs all the headstart it can get, not be complacent that some arbitrary "experts" will patronizingly "protect" them.

If you look at it as a PR stunt, it is almost certainly a good idea. If a bad actor can auto-generate text that is not really distinguishable from something written by a human, how does a community with open membership (eg, HN) protect itself? I imagine this technology will enable interesting new attacks against online communities; we havn't seen that for a while. OpenAI are extremely sensible to draw attention to th…

I’m not entirely sure that that bad actor would get any more scalablity form it than from a Mechanical Turk farm, at least as far as impact goes.

It seem that as far as information warfare goes “less is more” works quite well and they rely on targeted people to spread the news for them.

When you want to drive an agenda you don’t need unique 100,000 comments you need a good copy pasta.

Overall I’m sick of this dramatization of the AI catastrophe until there will be a proven path with agency for it to actually operate in the real world.

A chat bot isn’t a threat to anyone even if it turns homicidle.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#87

Post by Cacti »

repolfx wrote: post #80
roenxi wrote: post #54

Earlier quoted context omitted.

If you look at it as a PR stunt, it is almost certainly a good idea. If a bad actor can auto-generate text that is not really distinguishable from something written by a human, how does a community with open membership (eg, HN) protect itself? I imagine this technology will enable interesting new attacks against online communities; we havn't seen that for a while. OpenAI are extremely sensible to draw attention to th…

I'm not sure it has much in the way of implications. There is no real profit to be made by generating realistic looking text. Spammers don't work that way, spammers haven't cared about realistic looking text for years. Nor have spam filters cared much about text for a long time, exactly because it's so easy to randomise. Anti-spam is not a good reason to hold back on language generation models, in my view. As for HN,…

You’re fooling yourself if you think there are no significant uses of text generation. Fake news, propaganda, advertising, fake reviews, fake everything. Fabricated email from friends family and colleagues. Whole online communities fabricated out of whole cloth. It is a weapon, and a powerful one.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#88

Post by Eli_P »

When a bug is caught on your palm, it pretends to be a dead bug. When a moose is scared, it plays dead moose. When AI wants to fool a human or a captcha filter, it impersonates a human.

Only when a human wants to fool a human, it impersonates whatever possible but a human, then suddenly charges a shitload of ape shit, and then behaves like it never happened.

Without a decent natural language translation or automatic reasoning, which they have not, looks like N-gram where N equals to number of words in language corpus.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#89

Post by avinium »

pas wrote: post #72
roenxi wrote: post #54

Earlier quoted context omitted.

If you look at it as a PR stunt, it is almost certainly a good idea. If a bad actor can auto-generate text that is not really distinguishable from something written by a human, how does a community with open membership (eg, HN) protect itself? I imagine this technology will enable interesting new attacks against online communities; we havn't seen that for a while. OpenAI are extremely sensible to draw attention to th…

But ... it's not novel. We could already generate convincing gibberish years ago. Now the novelty is that this can be better targeted. But even simple Markov-chain based text generators were good enough to fool people for a bit. And there was always people that had too much free time to write. A lot. (See for example the crackpots and conspiracy theorists that bombard physics forums. See the 9/11, Zeitgeists, etc. mo…

The OpenAI samples are leaps and bounds ahead of traditional Markov-chain generated text. I don't think you can compare the two. It's the fluency and plausibility that gives pause around a public release.

I agree with your last point though - it falls into the same category as paid Russian trolls. I think that's exactly why they were hesitant to release the pre-trained models - they didn't want to make it easier/cheaper for a bad actor to replicate the 2016 election.

It remains to be seen whether their decision will make an iota of a difference. But I understand their motivation.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#90

Post by skybrian »

sgt101 wrote: post #71
moconnor wrote: post #43

Earlier quoted context omitted.

Actually this shows why OpenAI matters. Google have been training and refining Transformer architectures for years; how unlikely is it nobody tried training a language model at this scale or larger with similar results? Yet from Google we heard nothing. Which is the optimal decision for them - they only lose by blowing the whistle.

A lot of people have results similar to this - but most people generating a paragraph of slightly_weird_but_plausible_if_you_read_quickly text using a primped version of BERT one time out of 25 regarded it as more or less pointless. But journalists don't. This would be ok if this is the first time that anyone had a media go wild over AI story. But actually this has happened 10000 times this year already.

Seems like the way it worked is that the blog post was discussed here and on Twitter and many people thought it was interesting. Then some journalists picked it up and wrote about it.

That much is nothing out of the ordinary. It is interesting (at least to those of us who aren't natural language researchers) so why shouldn't we talk about it? Why shouldn't journalists write about it?

Inevitably their mildly controversial decision to hold some data back got a lot of people discussing whether it was necessary. Which is also perfectly okay.

So, in the end, the complaint is just about why people don't have smarter takes on things. I don't know what to tell you; that's just how social media works sometimes.

Post reply on HN