21–30 of 60 posts
Earlier quoted context omitted.
Wolfram can handle it: http://www.wolframalpha.com/input/?i=2nd+biggest+city+in+Cal...
That misses the key qualifier: "near a river". The challenge there is what is "near", and what is a "river"?
I was playing with it, had to go to a meeting and forgot I'd modified the question.
I learned about machine learning way after I learned mathematics, so it always amused me that back propagation = chain rule = forward differentiation = adjoint differentiation and that different disciplines have different words for what is just the chain rule.
These two comments seem to contradict each other: "we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like." "...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. " So w…
In my experience there are very few domains within Machine Learning where you don't need to be an expert in the field to yield useful conclusions out of the data.
Even if you have a high-level conceptual understanding of the statistical methods, tuning the parameters to yield something relevant, or much more so, adapting existing algorithms to meet your needs requires some pretty serious dedication to the field.
> When you have large amounts of data, your appetite for hypotheses tends to get even larger. And if it’s growing faster than the statistical strength of the data, then many of your inferences are likely to be false. They are likely to be white noise. It's actually worse than that. What I see is that when companies have the ability to store and "analyze" large amounts of data, their appetite for data tends to increas…
This is not necessarily a bad thing. Take the domain of application performance management. You're collecting hundreds of thousands of metrics from all over the place, OS, network, middleware, end user. Occasionally there is a performance problem that is non-obvious. You go through the obvious metrics and find nothing. It is a great thing at this point to just throw all this data at some algorithm and let it come back to you with "metric X, Y, Z looks related". This gives me some hypothesis I can go check that I would probably never have thought of on my own. And I have a direct way of verifying if it was a correct hypothesis: oh, it looks like there's 2 disks in this cluster, 1 is running at 100% the other at 0% so the overall utilization only shows 50%, I didn't think that was a problem. Investigate. Oh this disk has compression enabled, the other doesn't, turn it off, the application runs fast now.
>>Another example of a good language problem is question answering, like “What’s the second-biggest city in California that is not near a river?” If I typed that sentence into Google currently, I’m not likely to get a useful response. So I typed that in google just to see and indeed I got nothing. I guess their [1]knowledge graph still has a long way to go. [1 http://www.google.com/insidesearch/features/search/knowle…
For example, you can set a query "what's the 2nd biggest city in CA not near the river that has weather same as Seattle and is not among the top 500 cities in US".
As you can see generalized query would literally require system to create a program on its own. If we can do this, we would not need programmers and very likely it would be same breakthrough as practically unlimited supply of energy.
I don't think that anybody in the research community (except for maybe an occasional crazy) believes that neural networks have any biological significance beyond inspiration. NIPS (Neural Information Processing Systems) has been a reputable venue for work in statistics for some years now with no confusion over the idea that "Neural" does not mean a precise (or even imprecise) imitation of biological neurons.
This feedback loop explains a great chunk of why we on HN spend so much time knit-picking through stories on e.g. Wired. What we read is not so much "reporting," but designs-by-committee of researchers doing things they think the public wants/needs and reporters bending stories toward what they think the public wants and needs.
His comments are way off the mark. The recent advances in neural network training are not strictly due to convolutional neural networks, but rather the discovery that gradient descent works remarkably well on training multilayer neural networks when using modern hardware. All of the best performing pattern recognition techniques in speech, image recognition, and natural language processing now utilize "neural network…
"Well, I want to be a little careful here. I think it’s important to distinguish two areas where the word neural is currently being used.
One of them is in deep learning. And there, each “neuron” is really a cartoon. It’s a linear-weighted sum that’s passed through a nonlinearity. Anyone in electrical engineering would recognize those kinds of nonlinear systems. Calling that a neuron is clearly, at best, a shorthand. It’s really a cartoon. There is a procedure called logistic regression in statistics that dates from the 1950s, which had nothing to do with neurons but which is exactly the same little piece of architecture."