11–20 of 76 posts
Does anyone know how RAM-intensive deep learning is? If the answer is "not very," I think the GA144 might be a good candidate because it's a lot of CPU-capable (independently-branching) cores.
Well, it is definitely something but it being the "Breakthrough of the Decade" seems pretty unlikely to me (given my available evidence). I do not know well other examples beyond case of Automatic Speech Recognition, but since this case caused a lot of noise, I bet it is responsible for a reasonable chunk of the Deep learning "buzz". Here is my take about this. If you look at papers from Microsoft like Seide et al 20…
from the NYT article [1]: "The achievement was particularly impressive because the team decided to enter the contest at the last minute and designed its software with no specific knowledge about how the molecules bind to their targets. The students were also working with a relatively small set of data; neural nets typically perform well only with very large ones."
NNs in general have enjoyed lots of successful practical (commercial) applications in pattern recognition though they were sort of replaced in the "state-of-the-art" by SVMs in many cases until RBMs and DBNs came along. I agree with your caution for skepticism though, only time will tell how good DBNs are.
I think the black box criticism is BS for the most part. In some cases (google's search being a famous example) it might be great to have a human readable and tweakable solution (assuming you have the resources) but for something like recognising handwritten digits from images, not so much.
[1] http://www.nytimes.com/2012/11/24/science/scientists-see-adv...
Also, data science involves a lot more than building predictive models. In my experience >95% of effort goes into something other than building a model. In kaggle contests you usually concentrate on that edit: I haven't listened to the podcast yet (at work), my comment is more about the title.
Well, it is definitely something but it being the "Breakthrough of the Decade" seems pretty unlikely to me (given my available evidence). I do not know well other examples beyond case of Automatic Speech Recognition, but since this case caused a lot of noise, I bet it is responsible for a reasonable chunk of the Deep learning "buzz". Here is my take about this. If you look at papers from Microsoft like Seide et al 20…
There are nice and more balanced overviews here:
http://ufldl.stanford.edu/wiki/index.php/Deep_Networks:_Over...
I don't suppose someone has an alternative version somewhere?
Does anyone know how RAM-intensive deep learning is? If the answer is "not very," I think the GA144 might be a good candidate because it's a lot of CPU-capable (independently-branching) cores.
dont have any numbers for you but it's typically CPU-bound and not memory-bound. deep learning is often done using GPUs because of massive-parallelism.
Well, it is definitely something but it being the "Breakthrough of the Decade" seems pretty unlikely to me (given my available evidence). I do not know well other examples beyond case of Automatic Speech Recognition, but since this case caused a lot of noise, I bet it is responsible for a reasonable chunk of the Deep learning "buzz". Here is my take about this. If you look at papers from Microsoft like Seide et al 20…
I have no idea if it is the breakthrough of the decade, but I think deep learning isn't just taking a perceptron with many hidden layers and applying backpropagation to it, as you seem to say, all the interesting things about it you summarized as "fancy stuff" and "not making a big difference", without any context, references or arguments. I do not feel competent to discuss it as I have very little experience in this…
http://research.microsoft.com/apps/pubs/default.aspx?id=1531...
As I said I can only speak with more or less certainty regarding ASR. I am fairly sure that the success in ASR (with Google and MS embracing DNNs for ASR) contribute significantly to the mainstream impact of deep learning.
Well, it is definitely something but it being the "Breakthrough of the Decade" seems pretty unlikely to me (given my available evidence). I do not know well other examples beyond case of Automatic Speech Recognition, but since this case caused a lot of noise, I bet it is responsible for a reasonable chunk of the Deep learning "buzz". Here is my take about this. If you look at papers from Microsoft like Seide et al 20…
I've heard this argument ever since Norvig's Unreasonable Effectiveness of Data. While having a ton of data available is great, it has its limits. I believe you are overestimating the effectiveness of data (as, imo, Norvig did). And here specifically, it's not the case for the hype: from the NYT article [1]: "The achievement was particularly impressive because the team decided to enter the contest at the last minute…