Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unsupervised sentiment neuron

blog.openai.com

Post reply on HN
Print view View original

31–40 of 137 posts

Re: Unsupervised sentiment neuron

#31

Post by amelius »

emcq wrote: post #23

It's very difficult to understand what the contributions are here. From what I've read so far this feels more of a proposal for future research or a press release than advancing the state of the art. * Using large models trained on lots of data to provide the foundation for sample efficient smaller models is common. * Transfer learning, fine tuning, character RNNs is common. Were there any insights learned that give…

Also, first they write:

> We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment.

And then they write:

> We believe the phenomenon is not specific to our model, but is instead a general property of certain large neural networks that are trained to predict the next step or dimension in their inputs.

So they can't explain why a phenomenon is occurring, but they think that it generalizes to other contexts.

I find it all very unconvincing. Is this kind of writing common in the deep learning literature?

Re: Unsupervised sentiment neuron

#32

Post by kleiba »

emcq wrote: post #23

It's very difficult to understand what the contributions are here. From what I've read so far this feels more of a proposal for future research or a press release than advancing the state of the art. * Using large models trained on lots of data to provide the foundation for sample efficient smaller models is common. * Transfer learning, fine tuning, character RNNs is common. Were there any insights learned that give…

* advancing the state of the art

It says right at the top: "we get 91.8% accuracy versus the previous best of 90.2%" on a standard sentiment corpus. In addition, their method needs less training data than previous approaches.

* Were there any insights learned that give a deeper understanding of these phenomena?

The main appeal lies in the fact that a model trained on a (1) different and (2) very general task basically "in passing" also learned to predict sentiment (i.e., a specialized task that more or less arose from the domain the general model was trained on), and pretty much through a single neuron (out of the 4096 used). The authors speculate that this might be a general effect that could also be transferred to other prediction tasks.

Re: Unsupervised sentiment neuron

#33

Post by kleiba »

amelius wrote: post #31
emcq wrote: post #23

It's very difficult to understand what the contributions are here. From what I've read so far this feels more of a proposal for future research or a press release than advancing the state of the art. * Using large models trained on lots of data to provide the foundation for sample efficient smaller models is common. * Transfer learning, fine tuning, character RNNs is common. Were there any insights learned that give…

Also, first they write: > We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment. And then they write: > We believe the phenomenon is not specific to our model, but is instead a general property of certain large neural networks that are trained to predict the next step or dimension in thei…

Mind you, this is not a scientific publication but a blog post that has intentionally tried to adapt the tone to that medium, presumably to appeal to a wider audience.

Re: Unsupervised sentiment neuron

#34

Post by beaconstudios »

gallerdude wrote: post #19
haddr wrote: post #14

Earlier quoted context omitted.

We are getting better and better with automatic text generation. I wonder who will be the copyright owner of an AI-generated text, comments, songs, etc.?

A weird thought: at some point AI short stories may be far more profound than our own.

at the moment, AI short stories are derivative, so it's unlikely. They may well be better than the average, if trained on highly regarded works, but they're not completely novel.

Re: Unsupervised sentiment neuron

#35

Post by rspeer »

nl wrote: post #3

So char-by-char models is the next Word2Vec then. Pretty impressive results. It would be interesting to see how it performed for other NLP tasks. I'd be pretty interested to see how many neurons it uses to attempt something like stance detection. Data-parallelism was used across 4 Pascal Titan X gpus to speed up training and increase effective memory size. Training took approximately one month. Everytime I look at so…

Yeah, part of what let word2vec make such a splash that it became the one word embedding model everyone has heard of, is that the word2vec team released their model.

This is a really cool example OpenAI has, but I don't know why I should ultimately care about their character model more than anyone else's if all we've got is their description of how cool it is.

I hope OpenAI defies their reputation for closedness and releases the model.

Re: Unsupervised sentiment neuron

#36

Post by woodson »

phkahler wrote: post #27
derefr wrote: post #18

Earlier quoted context omitted.

This actually demonstrates something very interesting, I think: you can take an ML model trained with the "low-level prerequisite knowledge" of a subject, and then very quickly and easily teach it a high-level concept that relies on that knowledge. Which, now that I think about it, makes the human brain and its amazing adaptive general-game-playing abilities a bit less mysterious. Since we humans all have these huge…

Would that be why people raised in dysfunctional or abusive families have very deep rooted issues? Everything learned later in life seems more fluid, but some of those problematic attitudes or personality traits are extremely hard to change. Seems consistent with those parts being build in lower layers with a decades long training experience. And yet at a higher level things can be learned and changed fairly easily.

This is essentially linear algebra, not behavioural psychology. I believe that one shouldn't draw such broad conclusions from a 1% improvement on some evaluation dataset.

Re: Unsupervised sentiment neuron

#37

Post by RobertoG »

beaconstudios wrote: post #34
gallerdude wrote: post #19

Earlier quoted context omitted.

A weird thought: at some point AI short stories may be far more profound than our own.

at the moment, AI short stories are derivative, so it's unlikely. They may well be better than the average, if trained on highly regarded works, but they're not completely novel.

As they say: “When you take stuff from one writer it’s plagiarism, but when you take from many writers it’s called research.”

Re: Unsupervised sentiment neuron

#38

Post by ericjang »

Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still helpful for a non-ML audience to get a better understanding of the high-level motivation behind the work. The OpenAI folks are trying to educate the broader public as well, not just ML/AI researchers.

Imagine if this discovery were made by some undergraduate student who had little experience in the traditions of how ML benchmark experiments are done, or was just starting out her ML career. Would we be just as critical?

As a researcher, I like seeing shorter communications like these, as it illuminates the thinking process of the researcher. Read ML papers for the ideas, not the results :)

I personally don't mind blog posts that have a bit of hyped-up publicity. It's thanks to groups like DeepMind and OpenAI that have captured public imagination on the subject and accelerated such interest in prospective students in studying ML + AI + robotics. If the hype is indeed unjustified, then it'll become irrelevant in the long-term. One caveat is that researchers should be very careful to not mislead reporters who are looking for the next "killer robots" story. But that doesn't really apply here.

Re: Unsupervised sentiment neuron

#39

Post by ericjang »

Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still helpful for a non-ML audience to get a better understanding of the high-level motivation behind the work. The OpenAI folks are trying to educate the broader public as well, not just ML/AI researchers.

Imagine if this discovery were made by some undergraduate student who had little experience in the traditions of how ML benchmark experiments are done.

I personally don't mind blog posts that have a bit of hyped-up publicity. It's thanks to groups like DeepMind and OpenAI that have captured public imagination on the subject and accelerated such interest in prospective students in studying ML + AI + robotics. If the hype is unjustified, then it'll become irrelevant in the long-term.

Post reply on HN