why did they do this character by character? Would word by word make sense? Other than punctuation I'm not seeing why specific characters are meaningful units.
Unsupervised sentiment neuron
101–110 of 137 posts
Re: Unsupervised sentiment neuron
#102This has amazing potential for use in sock puppet accounts.
Re: Unsupervised sentiment neuron
#103Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still helpful for a non-ML audience to get a better understanding of the high-level motivation behind the work. The OpenAI folks are trying to educate the broader public as well, not just ML/AI researchers. Imagine if this discovery were made by some undergraduate…
> next "killer robots" story ... doesn't really apply here. Are you sure about that? We're talking about a model/robot which understand sentiments, and can generate fake reviews to boost fake products. I can easily see this being picked up by the AI hype journalists. In fact, this model could even be used for nefarious purposes.
> I can easily see this being picked up by the AI hype journalists.
Surely you mean AI journalist bots that can generate articles about how AI review bots generate fake reviews?
Re: Unsupervised sentiment neuron
#104Earlier quoted context omitted.
From my little experience with the AI community, I think people in it love to obfuscate things. Any attempt to make a topic approachable, even if some of the details are lost, get smacked around. I face this every day in my Masters. If you don't already come with a knowledge of AI + Stats, you're on your own. The community, including the teachers, don't want to teach the mundane.
What does "knowledge of AI" mean to you?
I'm in the GA Tech program. Chose AI since I've never touched it. Every course has been terrible in this track. The lectures are awful. The readings take hours upon hours. The TAs pathologically refuse to answer guestions without channeling Confucius. Then I found old GA Tech videos and MIT open courseware. It's starting to make sense. I say all of this because intro to ai is highly rated. It can only get such a rating if it's coming from people that already know the material.
Re: Unsupervised sentiment neuron
#105I don't know, but this seems a bit hyped in places. They start with: > Our L1-regularized model matches multichannel CNN performance with only 11 labeled examples, and state-of-the-art CT-LSTM Ensembles with 232 examples. Hmm, that sounds pretty impressive. But then later you read: > We first trained a multiplicative LSTM with 4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chu…
isn't it impressive? They trained a network to predict the next character and when the finished they trained the last layer to predict sentiment with only few examples. The network was able to learn sentiment somehow in the first training phase without telling it! For me this open my mind to new opportunities when training deep learning. For example I can do the same for images: train a network to recognize objects a…
Immediate thought - the network didn't learn sentiment in the training phase, it just clustered the data. The last phase was so quick to train because it was labeling already-clustered data, so a few data points in each cluster was enough to make it 'obvious'.
Re: Unsupervised sentiment neuron
#106Earlier quoted context omitted.
What you describe is exactly what practitioners in the field have been doing for years. I think that's why the parent is a bit puzzled at the publication, as it's difficult to understand what's novel.
Yes, I agree with you but with a caveat. Semi-supervised learning is well known but has, I'll argue, recently fallen out of fashion in favor of throwing gallons more of labeled data at a really big neural net, crossing your fingers and hoping for the best. Usually, the neural net is either a really big conv-net with a novel architecture or a biLSTM with some elaboration on attention (which is actually closer to memor…
thanks for clarifying this- it isn't transfer learning at all, more like the techniques like, digging through LSTMs post hoc to find the neuron responsible for opening and closing quotation marks (insert karpathy youtube vid here), except for a more "high level" feature - in this case, sentiment.
Re: Unsupervised sentiment neuron
#107Earlier quoted context omitted.
But given the whole corpus of all human-written texts, couldn't they be as creative as us?
It depends on your philosophy on what defines creativity, and whether all creativity is derivative of observing others.
I don't have one, and I guess that's still somehow unsolved problem. But there are some works on creativity and AI. This is definitely an interesting space to observe.
Re: Unsupervised sentiment neuron
#108Earlier quoted context omitted.
What does "knowledge of AI" mean to you?
I'm in the GA Tech program. Chose AI since I've never touched it. Every course has been terrible in this track. The lectures are awful. The readings take hours upon hours. The TAs pathologically refuse to answer guestions without channeling Confucius. Then I found old GA Tech videos and MIT open courseware. It's starting to make sense. I say all of this because intro to ai is highly rated. It can only get such a rati…
Are you in the OMSCS or on-campus program? I'm currently enrolled in CS 6601 (Artificial Intelligence) via OMSCS, and while it's extremely difficult material that requires a lot of personal investment to master, I've found both TAs and other students to be as helpful as one could reasonably expect.
Re: Unsupervised sentiment neuron
#109Earlier quoted context omitted.
I'm in the GA Tech program. Chose AI since I've never touched it. Every course has been terrible in this track. The lectures are awful. The readings take hours upon hours. The TAs pathologically refuse to answer guestions without channeling Confucius. Then I found old GA Tech videos and MIT open courseware. It's starting to make sense. I say all of this because intro to ai is highly rated. It can only get such a rati…
Are you in the OMSCS or on-campus program? I'm currently enrolled in CS 6601 (Artificial Intelligence) via OMSCS, and while it's extremely difficult material that requires a lot of personal investment to master, I've found both TAs and other students to be as helpful as one could reasonably expect.
OMSCS. The TAs refuse to answer questions because it could help cheating. They won't review homework because it could help cheating. They've created an environment so full of fear that it's impossible to learn from mistakes. I'm surprised the slack channel is allowed.
Re: Unsupervised sentiment neuron
#110Earlier quoted context omitted.
I'm in the GA Tech program. Chose AI since I've never touched it. Every course has been terrible in this track. The lectures are awful. The readings take hours upon hours. The TAs pathologically refuse to answer guestions without channeling Confucius. Then I found old GA Tech videos and MIT open courseware. It's starting to make sense. I say all of this because intro to ai is highly rated. It can only get such a rati…
Are you in the OMSCS or on-campus program? I'm currently enrolled in CS 6601 (Artificial Intelligence) via OMSCS, and while it's extremely difficult material that requires a lot of personal investment to master, I've found both TAs and other students to be as helpful as one could reasonably expect.
[deleted]