191–200 of 227 posts
There's a nice talk by Yann LeCun where he goes on to explain really well why deep learning has such fast progress. [0] He goes on to explain how theory always comes later. I thought that information theory came before practice but turns out it also came after. (there was a bunch of heuristics for sending messages with teletypes) A nice example is Roman technological advances in architecture and materials that predat…
Earlier quoted context omitted.
I mean in the sense that using ML for a problem often requires just trying a dozen different modeling techniques, then a bunch of a hyper-parameter searching, then a bunch of stochastic tuning…
Oh. I see what you mean. Yeah, I guess by definition backwards propagation is trial-end-error. Huh, I never thought of it that way. Thanks for clarifying, I thought you were being saucy: my apologies for being snarky.
This article is dead-on, but I think it is missing a fairly large segment of where ML is actually working well: anomaly detection and industrial defect detection. While I agree that everyone was shocked, myself included, when we saw how well SSD and YOLO worked, the last mile problem is stagnating. What I mean is: 7 years ago I wrote an image pipeline for a company using traditional AI methods. It was extremely chall…
>The industry is stagnated for exactly the reasons brought up: we don't know how to squeeze out the last mile problem because NNs are EFFING HARD and research is very math heavy: e.g., it cannot be hacked by a Zuck-type into a half-assed product overnight, it needs to be carefully researched for years. This makes programmers sad, because by nature we love to brute force trial-and error our code, and homey don't play…
Earlier quoted context omitted.
It's just another tool. I have this saying: all machine learning is just clever tricks and techniques until someone learns how to brute force the solution. It's the same adoption/business technology tension that has existed since Frederick Taylor in the early 1900s or Vonnegut's Player Piano concept where they propose taking a recorder to automated human-adverse tasks by recording their movements. The hype is trying…
> I have this saying: all machine learning is just clever tricks ... It appears to me that you are missing the mark here, unless this is largely a definitional issue. Do you consider the foundations of ML to be a clever trick? Do you think human brains primarily learn by clever tricks? When a metaphor or saying falls apart with one more level of questioning, I would suggest it may be time to find a better metaphor.
For your second, I think that's part of the problem. Most people confuse how a human brain learns with what is actually running in a machine learning program. It's similar at some level, but not really doing the same thing at another.
This article is dead-on, but I think it is missing a fairly large segment of where ML is actually working well: anomaly detection and industrial defect detection. While I agree that everyone was shocked, myself included, when we saw how well SSD and YOLO worked, the last mile problem is stagnating. What I mean is: 7 years ago I wrote an image pipeline for a company using traditional AI methods. It was extremely chall…
I worked and built out a proof of concept industrial defect detection system recently, with a large focus on modern DNN architectures. We worked with a plant to curate a 30000+ multi-class defect dataset, many with varying lighting and environment conditions. As you said, modifying and parameter tuning NN is not always a hopeful endeavor. However, you can make significant gains to your models by going back to traditi…
We'd love to be able to work with a company for a few days, get the parameters set up right for our case, and then let them take the thousands of images. My company would easily pay $100K+ for such a data set.
Astronomy has entered a period of great stagnation. More and more grad students are investing huge amount of time and building larger and larger radio telescopes just to learn more about black hole formation and the properties of pulsars. Little consideration is given to how to more efficiently use inexpensive consumer telescopes purchased at Walmart! Where are the big discoveries? New planets in our solar system? A…
Earlier quoted context omitted.
When I was in grad school in 1999 studying AI the general consensus was that neural networks didn't work very well, and that you were better off with the more mathematically grounded methods like support vector machines. TD-Gammon was just about the only success story for neural networks, there hadn't been much else since 1992.
I don't know where that "general consensus" came from, because in 1998 Lecun clearly showed [1] convnets beating all other models, including SVMs, at image recognition. [1] http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf
Younger people don't realize there was strong bias against using neural networks in the late 90s up until Hinton's talk on NNs around 2007. I get the feeling we're going through the same thing where novel research is becoming ignored because everything must fit the deep learning paradigm to be noticed.