Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Trains Language Model, Mass Hysteria Ensues

approximatelycorrect.com

Post reply on HN
Print view View original

51–60 of 119 posts

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#51

Post by pishpash »

cma wrote: post #4
Permit wrote: post #3

> Namely, he argued that OpenAI is concerned that the technology might be used to impersonate people or to fabricate fake news. This seems to be a particularly weak argument to make. How is their model going to impersonate someone in a way that a human can not?

Cheaper cost to put out a bigger volume of content.

But cheaper cost for everyone else also, provided they own the tech. That seems an argument for wide distribution. (Do you want to be the lone human voice against the bots or do you want to have your own bots to amplify your voice?)

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#52

Post by namuol »

Barrin92 wrote: post #45
namuol wrote: post #21

Earlier quoted context omitted.

> not extraordinary compared to state of the art, open NLP research > misrepresents progress in the field Can you point me to some examples of unsupervised learning with similar results? Not asking for rhetorical purposes; I just genuinely was shocked by how compelling their results were, especially given this was unsupervised. > OpenAI's behaviour here smells of Gibsonesque 'anti-marketing' I don't disagree that the…

>Can you point me to some examples of unsupervised learning with similar results? Not asking for rhetorical purposes; I just genuinely was shocked by how compelling their results were, especially given this was unsupervised. Model wise this is just openAI's GPT with some very slight modification (laid out in the paper). Ilya has now commented in the thread and essentially made the same point, this is state of the art…

> Model wise this is just openAI's GPT with some very slight modification (laid out in the paper).

> The secrecy and controversy makes no sense if the model is open, even the methodology of data collection is laid out.

This is exactly why I found the results so compelling: It suggests that this technology is already accessible to some big players: The odds that a Big Corp. or govt agency has already begun using the technology are high, which is precisely why the public needs to start thinking about it.

I cannot know exactly why OpenAI chose to withhold the model, especially given how easy it would be to recreate, but even if we assume that OpenAI withheld the full model purely to drum up controversy, the controversy is justified, as it's very likely that this technology is already in the hands of a few big players.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#53

Post by chrinic83 »

xiphias2 wrote: post #12

Elon Musk was kicked out because he poached Andrej Karpathy from OpenAI to lead Autopilot. Anyways, it was worth it, Andrej is doing an amazing job, and OpenAI is still alive :)

> Anyways, it was worth it, Andrej is doing an amazing job, and OpenAI is still alive :)

Tesla does not even offer their full self driving package anymore. No coast to coast drive yet. Hard to say that's an amazing job.

OpenAI abandons their open source GitHub repos after a year, is now not releasing code, and is always in DeepMind's shadow. Alive, yes. Successful, no.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#54

Post by roenxi »

pishpash wrote: post #42
Cacti wrote: post #38

Earlier quoted context omitted.

> - The _whole point_ is that our model is not special and that other people can reproduce and improve upon what we did. We hope that when they do so, they too will reflect about the consequences of releasing their very powerful text generation models. If this is your whole point, then I think you are missing something fundamental. Implementing these models doesn't require reflection, or introspection, or any sort of…

Exactly. This is like holding up spam samples or how spammers operate from the spam detecting work. That side (and the cultural discussions) needs all the headstart it can get, not be complacent that some arbitrary "experts" will patronizingly "protect" them.

If you look at it as a PR stunt, it is almost certainly a good idea. If a bad actor can auto-generate text that is not really distinguishable from something written by a human, how does a community with open membership (eg, HN) protect itself? I imagine this technology will enable interesting new attacks against online communities; we havn't seen that for a while.

OpenAI are extremely sensible to draw attention to the fact that AI is approaching a boundary that has practical implications. It is good that everyone is being alerted that that boundary might be crossed at any time in the foreseeable future.

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#56

Post by pas »

avip wrote: post #35
ilyasut wrote: post #23

Ilya from OpenAI here. Here's our thinking: - ML is getting more powerful and will continue to do so as time goes by. While this point of view is not unanimously held by the AI community, it is also not particularly controversial. - If you accept the above, then the current AI norm of "publish everything always" will have to change - The _whole point_ is that our model is not special and that other people can reprodu…

I've just read i.e https://twitter.com/gdb/status/1096098366545522688 and even though it's "best of 25" (I guess cherry-picked by a human) - this is mind-blowing. I am actually having a very hard time believing this is legit generated text.

Why? It is pretty much a well juxtaposed mix of random internet comments. And it's the best of 25, which means the other 24 is even more regular internet banter noisy.

(This of course doesn't make it an amazing feat of computer engineering.)

The overarching narrative is great, but that's probably driven by the great antithesis supplied by the experimenter.

It'd be interesting to know how this works, what happens if less or more is given as thesis/antithesis/assignment, and after how much output it turns into gibberish (or repeats).

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#58

Post by bilbo0s »

Cacti wrote: post #38
ilyasut wrote: post #23

Ilya from OpenAI here. Here's our thinking: - ML is getting more powerful and will continue to do so as time goes by. While this point of view is not unanimously held by the AI community, it is also not particularly controversial. - If you accept the above, then the current AI norm of "publish everything always" will have to change - The _whole point_ is that our model is not special and that other people can reprodu…

> - The _whole point_ is that our model is not special and that other people can reproduce and improve upon what we did. We hope that when they do so, they too will reflect about the consequences of releasing their very powerful text generation models. If this is your whole point, then I think you are missing something fundamental. Implementing these models doesn't require reflection, or introspection, or any sort of…

>You are fooling yourself if you think your stance makes a single mote of difference in this arms race...

In fairness, if that's true, then no one has any need of her model.

More seriously speaking, why does anyone need, say, "training set x", or "model y", to make their implementation work? You don't. So I don't really understand why everyone is so worked up about not releasing this stuff? If you want to do it, do it. If not, don't. But there's no need to say, "I demand everyone do it, and I'll have a meltdown if they don't."

Re: OpenAI Trains Language Model, Mass Hysteria Ensues

#60

Post by sigil »

albntomat0 wrote: post #46
sigil wrote: post #29

Earlier quoted context omitted.

Ok but isn’t this the opposite of OpenAI’s “nukes are safer when multiple actors have them” strategy wrt AI? I’m also confused by the threat models earnestly put forth in your blog post. Are we really concerned about deep faking someone’s writing? The plain word already demands attribution by default: we look for an avatar, a handle, a domain name to prove the person actually said this.

> Ok but isn’t this the opposite of OpenAI’s “nukes are safer when multiple actors have them” strategy wrt AI? It seems more like the "nukes are safer when multiple rational state level actors have them", rather than anyone able to pull a git repo.

Yep. Maybe I misunderstood the subtler points of OpenAI’s “democratize AI” strategy, and this has been the plan all along. But AFAIK they haven’t put an “among a few rational state actors” asterisk on anything up until now.

Regardless, I agree with TFA that this is a silly and arbitrary time to yell “fire.” It’s PR.

Post reply on HN