Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unsupervised sentiment neuron

blog.openai.com

Post reply on HN
Print view View original

41–50 of 137 posts

Re: Unsupervised sentiment neuron

#41

Post by [deleted] »

kleiba wrote: post #33
amelius wrote: post #31

Earlier quoted context omitted.

Also, first they write: > We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment. And then they write: > We believe the phenomenon is not specific to our model, but is instead a general property of certain large neural networks that are trained to predict the next step or dimension in thei…

Mind you, this is not a scientific publication but a blog post that has intentionally tried to adapt the tone to that medium, presumably to appeal to a wider audience.

[deleted]

Re: Unsupervised sentiment neuron

#42

Post by aabajian »

I'm trying to understand this statement:

"The sentiment neuron within our model can classify reviews as negative or positive, even though the model is trained only to predict the next character in the text."

If you look closely at the colorized paragraph in their paper/website, you can see that the major sentiment jumps (e.g. from green to light-green and from light-orangish to red) occur with period characters. Perhaps the insight is that periods delineate the boundary of sentiment. For example:

I like this movie. I liked this movie, but not that much. I initially hated the movie, but ended up loving it.

The period tells the model that the thought has ended.

My question for the team: How well does the model perform if you remove periods?

Re: Unsupervised sentiment neuron

#43

Post by gdb »

rspeer wrote: post #35
nl wrote: post #3

So char-by-char models is the next Word2Vec then. Pretty impressive results. It would be interesting to see how it performed for other NLP tasks. I'd be pretty interested to see how many neurons it uses to attempt something like stance detection. Data-parallelism was used across 4 Pascal Titan X gpus to speed up training and increase effective memory size. Training took approximately one month. Everytime I look at so…

Yeah, part of what let word2vec make such a splash that it became the one word embedding model everyone has heard of, is that the word2vec team released their model. This is a really cool example OpenAI has, but I don't know why I should ultimately care about their character model more than anyone else's if all we've got is their description of how cool it is. I hope OpenAI defies their reputation for closedness and…

Yep weights will be up soon!

EDIT: in fact, weights were up at launch: https://github.com/openai/generating-reviews-discovering-sen...

Re: Unsupervised sentiment neuron

#44

Post by haddr »

beaconstudios wrote: post #34
gallerdude wrote: post #19

Earlier quoted context omitted.

A weird thought: at some point AI short stories may be far more profound than our own.

at the moment, AI short stories are derivative, so it's unlikely. They may well be better than the average, if trained on highly regarded works, but they're not completely novel.

But given the whole corpus of all human-written texts, couldn't they be as creative as us?

Re: Unsupervised sentiment neuron

#45

Post by happycube »

beaconstudios wrote: post #34
gallerdude wrote: post #19

Earlier quoted context omitted.

A weird thought: at some point AI short stories may be far more profound than our own.

at the moment, AI short stories are derivative, so it's unlikely. They may well be better than the average, if trained on highly regarded works, but they're not completely novel.

At the moment RNN's can't remember context, so they can make stuff that looks correct, but only on the surface.

I think that'll change, eventually...

Re: Unsupervised sentiment neuron

#46

Post by phkahler »

woodson wrote: post #36
phkahler wrote: post #27

Earlier quoted context omitted.

Would that be why people raised in dysfunctional or abusive families have very deep rooted issues? Everything learned later in life seems more fluid, but some of those problematic attitudes or personality traits are extremely hard to change. Seems consistent with those parts being build in lower layers with a decades long training experience. And yet at a higher level things can be learned and changed fairly easily.

This is essentially linear algebra, not behavioural psychology. I believe that one shouldn't draw such broad conclusions from a 1% improvement on some evaluation dataset.

I was talking about the ability to use thin layers over top of broad learning. If neural networks are a realistic analogy at all, I think it fits. The roots of all this ML stuff is not straight from linear algebra even though much of the math is.

Re: Unsupervised sentiment neuron

#47

Post by rspeer »

gdb wrote: post #43
rspeer wrote: post #35

Earlier quoted context omitted.

Yeah, part of what let word2vec make such a splash that it became the one word embedding model everyone has heard of, is that the word2vec team released their model. This is a really cool example OpenAI has, but I don't know why I should ultimately care about their character model more than anyone else's if all we've got is their description of how cool it is. I hope OpenAI defies their reputation for closedness and…

Yep weights will be up soon! EDIT: in fact, weights were up at launch: https://github.com/openai/generating-reviews-discovering-sen...

Sorry for my pessimistic outlook, then! Thanks.

Re: Unsupervised sentiment neuron

#48

Post by visarga »

phkahler wrote: post #27
derefr wrote: post #18

Earlier quoted context omitted.

This actually demonstrates something very interesting, I think: you can take an ML model trained with the "low-level prerequisite knowledge" of a subject, and then very quickly and easily teach it a high-level concept that relies on that knowledge. Which, now that I think about it, makes the human brain and its amazing adaptive general-game-playing abilities a bit less mysterious. Since we humans all have these huge…

Would that be why people raised in dysfunctional or abusive families have very deep rooted issues? Everything learned later in life seems more fluid, but some of those problematic attitudes or personality traits are extremely hard to change. Seems consistent with those parts being build in lower layers with a decades long training experience. And yet at a higher level things can be learned and changed fairly easily.

Could be possible. For example, when a child is deprived of any human and humane contact, they end up with a very very impoverished linguistic system as well as delayed development of all cognitive abilities. Could be because the lack the unsupervised training phase.

Re: Unsupervised sentiment neuron

#49

Post by scythe »

phkahler wrote: post #46
woodson wrote: post #36

Earlier quoted context omitted.

This is essentially linear algebra, not behavioural psychology. I believe that one shouldn't draw such broad conclusions from a 1% improvement on some evaluation dataset.

I was talking about the ability to use thin layers over top of broad learning. If neural networks are a realistic analogy at all, I think it fits. The roots of all this ML stuff is not straight from linear algebra even though much of the math is.

You're playing it extremely fast and loose with concepts like "low-level prerequisite knowledge" and how exactly does something "rel[y] on that knowledge", though. These aren't physical quantities like temperature where we -- as a species -- have the massive amount of low-level prerequisite knowledge that allows us to make rapid high level judgments that rely on that knowledge. The previous sentence is an example of how easy this reasoning is to abuse.

Re: Unsupervised sentiment neuron

#50

Post by itchyjunk »

I would imagine stuff like sarcasm is still out of reach though. It seems hard for humans to understand it in text based communication. Also using anything out of the standard sentimental model might throw it off. "This product is as good as (where product x has been known to perform bad." I am just trying to think of scenarios where a sentimental model would fail.

Sentimental neuron sounds fascinating too. I didn't realize individual neurons could be talked about or understood outside of the concept of the NN. I am thinking in terms of "black box" its often referenced to in some articles.

Since one of the research goal for openai is to train language model on jokes[0], I wonder how this neuron would perform with a joke corpus.

----------------------------

[0] https://openai.com/requests-for-research/#funnybot

Post reply on HN