61–70 of 137 posts
> In practice, neural networks use only two or three layers... The famous AlexNet [1] that blew away the ImageNet competition in 2012 contained 8 layers; more recent networks have even more. [1] http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf
There was a great article called The Space Doctor's Big Idea , published in the New Yorker a few months back. It explained Einstein's theories in the top 1000s English words used in America. While sone might wave their hands and call this an eli5, it was quite well done and between that and some xkcd comics I was able to learn enough to ubderstand how orbit works and a few other awesome facts. I am interested in find…
Neural networks is advanced curve fitting -- That's why. It isn't really all that magical or mysterious.
It really kinda is. In the Ilya Sutskever Talking Machines podcast[1] he describes exactly how it is "magic". He talks about how there is no theoretical basis to think that a deep neural network should converge, and prior to around 2006 the accepted wisdom was that networks deep enough to outperform other methods of machine learning were useless because they couldn't be trained. And then they discovered how to initia…
I can chime in for the theoretical computer scientists. Deep learning (in its simplest form) corresponds to the class of circuits whose gates are linear threshold functions. Our primary goal with such functions is not to show what problems can be solved by small circuits using linear threshold gates, but what problems cannot be solved with such circuits. Until last week [1], it was an open problem whether every funct…
Actually, there's been a few papers showing solid theoretical progress on understanding what structure in data deep networks are learning to represent. https://github.com/gregversteeg/CorEx http://arxiv.org/abs/1406.1222 http://arxiv.org/abs/1410.7404 Disclaimer: I reinvented the idea about a month or two ago, and ran into the existing papers when googling for joint entropy estimators after making some interesting gr…
The whole idea of "simplest model that explains most of the data" has always been very appealing to me. The concept is closely tied to reproducing kernel Hilbert spaces, which have recently experienced a revival in interest due to the representer theorem.
Earlier quoted context omitted.
I'm not sure how old you are, but what exactly are you expectations when you state Data science is improving, but you might be surprised how slowly. We have machines that can categorise pictures better that humans. In 2011 that seemed completely impossible.
Age is unrelated to wisdom and I'm talking about the full experience. A Google Image search for "Wonder Wheel" (the famous Coney Island Ferris Wheel) shows this spoked diagram within the first page of results: http://searchengineland.com/figz/wp-content/seloads/2011/07/... Also this year, Google Photos classified black people as gorillas. http://www.usatoday.com/story/tech/2015/07/01/google-apologi... Consumers are r…
To take the discussion on a slight tangent, how uncommon is this phenomenon? An applied tool works really well but nobody knows why. I can give another example from the domain of formal verification: SAT solvers which are at the core of most modern verification/synthesis tools. You can download open source SAT solvers today that work spectacularly well on "real" SAT instances with millions and millions of variables a…
There's also the intriguing possibility that "natural instances" however one might define them arise through some complicated process that ends up sampling from easy SAT instances. I think of it as analogous to the Benford's law (see: https://en.wikipedia.org/wiki/Benford%27s_law#Explanations) except that we do not have nearly as good explanations for SAT solvers.
It is also highly likely that MOST instances of SAT are easy but that doesn't preclude it from being NP hard, nor does it stop us from easily constructing hard instances. (Such things are absolutely necessary for cryptography as an example.)
Earlier quoted context omitted.
Age is unrelated to wisdom and I'm talking about the full experience. A Google Image search for "Wonder Wheel" (the famous Coney Island Ferris Wheel) shows this spoked diagram within the first page of results: http://searchengineland.com/figz/wp-content/seloads/2011/07/... Also this year, Google Photos classified black people as gorillas. http://www.usatoday.com/story/tech/2015/07/01/google-apologi... Consumers are r…
That spoked diagram's image is named "google-wonder-wheel" and is linked from an article referring to Google's Wonder Wheel. That doesn't seem like a spurious result to me...
The gorilla example is more to the point. Google pushed a quick fix and couldn't fix it. So they blocked the gorilla tag altogether. Why? Because a data scientist couldn't figure out how to fix the problem properly and deploy a solution. To me, that says quite a bit about the current realities of machine learning systems.
As for "spectacularly well" -- well, the person behind the curtain wiggling the levers retains a lot of influence. Garbage in, garbage out, remember? I'm reminded of the time Google Translate autodetected "Gesundheit" as Spanish. And Gmail kindly offering to translate "hahaha" from Portuguese, putting an ad for coconuts next to it. Data science is improving, but you might be surprised how slowly. Especially in the co…
I'm not sure how old you are, but what exactly are you expectations when you state Data science is improving, but you might be surprised how slowly. We have machines that can categorise pictures better that humans. In 2011 that seemed completely impossible.
In addition to "We need a cluster for deep learning", the second most popular mostly untrue thing I hear is "We have no idea how neural networks learn". However, there are many papers that explore various ways to make a network learn, and they keep improving on performance, suggesting they're on to something . There are also many papers that discuss possible theoretical implications of experimental results. But what…
I can chime in for the theoretical computer scientists. Deep learning (in its simplest form) corresponds to the class of circuits whose gates are linear threshold functions. Our primary goal with such functions is not to show what problems can be solved by small circuits using linear threshold gates, but what problems cannot be solved with such circuits. Until last week [1], it was an open problem whether every funct…
Actually, there's been a few papers showing solid theoretical progress on understanding what structure in data deep networks are learning to represent. https://github.com/gregversteeg/CorEx http://arxiv.org/abs/1406.1222 http://arxiv.org/abs/1410.7404 Disclaimer: I reinvented the idea about a month or two ago, and ran into the existing papers when googling for joint entropy estimators after making some interesting gr…
10.1364/JOSAA.9.000516 Shiro Usui, Shigeki Nakauchi, and Masae Nakano, "Reconstruction of Munsell color space by a five-layer neural network," J. Opt. Soc. Am. A 9, 516-520 (1992)