Machine Learning has become more and more like archaeology after people start saying "empirically" more and only provide a single or limited datasets.
Unsupervised sentiment neuron
81–90 of 137 posts
Re: Unsupervised sentiment neuron
#82(Apologies for the slightly incoherent post below)
I've been noticing a lot of work that digs into ML model internals (as they've done here to find the sentiment neuron) to understand why they work or use them to do something. Let me recall interesting instances of this:
1. Sander Dieleman's blog post about using CNNs at Spotify to do content-based recommendations for music. He didn't write about the system performance but collected playlists that maximally activated each of the CNN filters (early layer filters picked up on primitive audio features, later ones picked up on more abstract features). The filters were essentially learning the musical elements specific to various subgenres.
2. The ELI5 - Explain Like I'm Five - Python Library. It explains the outputs of many linear classifiers. I've used it to explain why a text classifier was given a certain prediction: it highlights features to show how much or little they contribute to the prediction (dark red for negative contribution, dark green for positive contribution).
3. FairML: Auditing black-box models. Inspecting the model to find which features are important. With privacy and security concerns too!
Since deep learning/machine learning is very empirical at this stage, I think improvements in instrumentation can lead to ML/DL being adopted for more kinds of problems. For example: chemical/biological data. I'd be highly curious to what new ways of inspecting such kinds of data would be insightful (we can play audio input that maximally active filters for a music-related network, we can visualize what filters are learning in an object detection network, etc.)
Re: Unsupervised sentiment neuron
#83I would imagine stuff like sarcasm is still out of reach though. It seems hard for humans to understand it in text based communication. Also using anything out of the standard sentimental model might throw it off. "This product is as good as (where product x has been known to perform bad." I am just trying to think of scenarios where a sentimental model would fail. Sentimental neuron sounds fascinating too. I didn't…
>>>Sentimental neuron sounds fascinating too. I didn't realize individual neurons could be talked about or understood outside of the concept of the NN. I am thinking in terms of "black box" its often referenced to in some articles.
Yes, I agree. I recall seeing such individual neuron analysis before in Karpathy's "The Unreasonable Effectiveness of Recurrent Neural Networks". He takes a char-rnn that was training to predict the next character for source code and finds neurons that have learned to do paranthesis/bracket opening/closing.
Re: Unsupervised sentiment neuron
#84Earlier quoted context omitted.
Yeah, part of what let word2vec make such a splash that it became the one word embedding model everyone has heard of, is that the word2vec team released their model. This is a really cool example OpenAI has, but I don't know why I should ultimately care about their character model more than anyone else's if all we've got is their description of how cool it is. I hope OpenAI defies their reputation for closedness and…
> OpenAI defies their reputation for closedness Does not compute.
The 90's definition of Open? ;)
Although in this case, they did post the weights quickly.
Re: Unsupervised sentiment neuron
#85Can someone explain what is "unsupervised" about this? I'm guessing this is what confuses me most. I think this work is interesting, although when you think about it, it's kind of normal that the model converges to a point where there is a neuron that indicates whether the review is positive or negative. There are probably a lot of other traits that can be found in the "features" layer as well. There are probably neu…
I agree the title is confusing. As far as I understand everything is unsupervised except for the sentiment neuron. The paper itself is actually a better read.
Re: Unsupervised sentiment neuron
#86[deleted]
Re: Unsupervised sentiment neuron
#87Can someone explain what is "unsupervised" about this? I'm guessing this is what confuses me most. I think this work is interesting, although when you think about it, it's kind of normal that the model converges to a point where there is a neuron that indicates whether the review is positive or negative. There are probably a lot of other traits that can be found in the "features" layer as well. There are probably neu…
I wouldn't expect that the neurons are orthogonal on a set of features which we find interesting (sentiment, geographical location). They could be bound up in some other basis of features that we do not find interesting. Other people do not expect this because there are papers about how to incentivize neurons to correspond to interesting features.
>> Other people do not expect this because there are papers about how to incentivize neurons to correspond to interesting features.
Could you clarify that statement? Are you saying that it was unusual for this group to find such a neuron? Also, I did not know that there are papers on how to incentivize neurons to correspond to interesting features. Could you please give me some references on those?
Re: Unsupervised sentiment neuron
#88Can someone explain what is "unsupervised" about this? I'm guessing this is what confuses me most. I think this work is interesting, although when you think about it, it's kind of normal that the model converges to a point where there is a neuron that indicates whether the review is positive or negative. There are probably a lot of other traits that can be found in the "features" layer as well. There are probably neu…
I agree the title is confusing. As far as I understand everything is unsupervised except for the sentiment neuron. The paper itself is actually a better read.
Yes, it's coincidental rather than unsupervised... The fact that to have a good "next character" prediction, you need to know about the "mindset" of the author makes sense. Especially in the context of product reviews.
Had they created a next-move predictor for chess, they wouldn't have been surprised to find a neuron representing the aggressiveness of the player.
It's a good result on its own but the word "unsupervised" is a bit annoying.
Re: Unsupervised sentiment neuron
#89Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still helpful for a non-ML audience to get a better understanding of the high-level motivation behind the work. The OpenAI folks are trying to educate the broader public as well, not just ML/AI researchers. Imagine if this discovery were made by some undergraduate…
From my little experience with the AI community, I think people in it love to obfuscate things. Any attempt to make a topic approachable, even if some of the details are lost, get smacked around. I face this every day in my Masters. If you don't already come with a knowledge of AI + Stats, you're on your own. The community, including the teachers, don't want to teach the mundane.
What does "knowledge of AI" mean to you?
Re: Unsupervised sentiment neuron
#90Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still helpful for a non-ML audience to get a better understanding of the high-level motivation behind the work. The OpenAI folks are trying to educate the broader public as well, not just ML/AI researchers. Imagine if this discovery were made by some undergraduate…
Is it wrong to be critical of research? Back in my previous life of doing basic research I scrutinized papers left and right.
http://karpathy.github.io/2015/05/21/rnn-effectiveness/ towards the end has similar methodology and is 1.5 years old.
Hype is an interesting thing especially when it comes from laymen.