Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Trains Language Model, Mass Hysteria Ensues

approximatelycorrect.com

Post reply on HN
Print view View original

31–40 of 119 posts

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#31

Post by malux85 »

ilyasut wrote: post #23

Ilya from OpenAI here. Here's our thinking: - ML is getting more powerful and will continue to do so as time goes by. While this point of view is not unanimously held by the AI community, it is also not particularly controversial. - If you accept the above, then the current AI norm of "publish everything always" will have to change - The _whole point_ is that our model is not special and that other people can reprodu…

So a small number of individuals decided what's best for everybody?

How is that open?

How is that not centralization of power?

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#32

Post by HALtheWise »

SubiculumCode wrote: post #13

To what extent is this not just finding text samples written in its training sample and regurgitating it near verbatim?? -Non ml guy

If you look at their paper, section 4 is entirely devoted to this question. They present compelling evidence that it is generating original content, the simplest of which is it's ability to write coherently about ridiculous things like talking unicorns that nobody has ever written about in the training set.

https://d4mucfpksywv.cloudfront.net/better-language-models/l...

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#33

Post by IanCal »

m0zg wrote: post #9
namuol wrote: post #7

Earlier quoted context omitted.

First, it's clearly the goal of OpenAI to bring more public attention to advances in the field, specifically to help voters and policymakers consider potential ramifications well in advance of any "truly" groundbreaking work before it's too late. Of course they're "hyping" this technology. Secondly, have you seen the results? I was dumbfounded and fascinated. I spent hours reading the samples. Maybe I'm just out of t…

I have seen the results and I don't get why people think this is any more dangerous than journalists who selectively report to fit a predetermined agenda or make shit up on the spot. Which, today, is a lot of them.

You can't do that for near 0 cost though, nor generate a different story per user on the fly.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#34

Post by onurcel »

ilyasut wrote: post #23

Ilya from OpenAI here. Here's our thinking: - ML is getting more powerful and will continue to do so as time goes by. While this point of view is not unanimously held by the AI community, it is also not particularly controversial. - If you accept the above, then the current AI norm of "publish everything always" will have to change - The _whole point_ is that our model is not special and that other people can reprodu…

Deciding if feeding the media with fear was worth the attention you will get wasn't easy, ha. Let me tell you, you are the shame of the profession.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#35

Post by avip »

ilyasut wrote: post #23

Ilya from OpenAI here. Here's our thinking: - ML is getting more powerful and will continue to do so as time goes by. While this point of view is not unanimously held by the AI community, it is also not particularly controversial. - If you accept the above, then the current AI norm of "publish everything always" will have to change - The _whole point_ is that our model is not special and that other people can reprodu…

I've just read i.e https://twitter.com/gdb/status/1096098366545522688 and even though it's "best of 25" (I guess cherry-picked by a human) - this is mind-blowing. I am actually having a very hard time believing this is legit generated text.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#36

Post by roywiggins »

Permit wrote: post #3

> Namely, he argued that OpenAI is concerned that the technology might be used to impersonate people or to fabricate fake news. This seems to be a particularly weak argument to make. How is their model going to impersonate someone in a way that a human can not?

You might be able to tailor things to individual people very specifically based on what their views are and what might push their buttons. Like spearfishing but for propaganda. Not with this exact tool, you'd probably need some more knobs, but a similar one. This would be impractical to do at scale without computer assistance.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#37

Post by Eliezer »

It seems disingenuous that this article fails to quote examples of GPT-2’s stunning results, or give any contrasting results from BERT to support the claim that this is all normal and expected progress.

Like many, I was viscerally shocked that the results were possible, the potential to further wreck the Internet seemed obvious, and an extra six months for security actors to prepare a response seemed like normal good disclosure practice. OpenAI warned everyone of an “exploit” in which text humans can trust to be human-generated, and then announced they would hold off on publishing the exploit code for 6 months. This is normal in computer security and I’m taken aback at how little the analogy seems to be appreciated.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#38

Post by Cacti »

ilyasut wrote: post #23

Ilya from OpenAI here. Here's our thinking: - ML is getting more powerful and will continue to do so as time goes by. While this point of view is not unanimously held by the AI community, it is also not particularly controversial. - If you accept the above, then the current AI norm of "publish everything always" will have to change - The _whole point_ is that our model is not special and that other people can reprodu…

> - The _whole point_ is that our model is not special and that other people can reproduce and improve upon what we did. We hope that when they do so, they too will reflect about the consequences of releasing their very powerful text generation models.

If this is your whole point, then I think you are missing something fundamental. Implementing these models doesn't require reflection, or introspection, or any sort of ethical or moral character whatsoever; and even if it did, all that will happen eventually is someone (without the technical background) will simply throw a lot of money at someone else (with the technical background, but who needs to, you know, eat, and pay rent, and so on) to implement it. You are fooling yourself if you think your stance makes a single mote of difference in this arms race.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#39

Post by arcticfox »

HALtheWise wrote: post #32
SubiculumCode wrote: post #13

To what extent is this not just finding text samples written in its training sample and regurgitating it near verbatim?? -Non ml guy

If you look at their paper, section 4 is entirely devoted to this question. They present compelling evidence that it is generating original content, the simplest of which is it's ability to write coherently about ridiculous things like talking unicorns that nobody has ever written about in the training set. https://d4mucfpksywv.cloudfront.net/better-language-models/l...

The talking unicorns piece was shockingly good. That is at least as coherent of a news story than the average human could easily invent about it.

Reading that piece gives me the same weird feeling as watching AlphaStar playing through a StarCraft game.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#40

Post by pishpash »

Permit wrote: post #5
cma wrote: post #4

Earlier quoted context omitted.

Cheaper cost to put out a bigger volume of content.

Is volume really what dictates whether or not you can impersonate someone? It's never seemed that way to me.

More to the point, is volume what counts as danger? All these deepfake risks boil down to online (for now) sock puppetry. We've been dealing with that for the whole life of the internet. The only reason it's even a problem in recent years is the growth of uninnoculated masses who haven't been on the internet that long, and positive feedback recommendation bots. That seems a qualitative issue not a quantitative one.
Post reply on HN