Earlier quoted context omitted.
I'm not sure about that. The new GAN models over the past 2-3 months, like LS-GAN or WGAN, all seem to train much more stably. I've beaten up on WGAN with all sorts of strange tweaks and hyperparameter settings and while it may not work well, it's never catastrophically diverged on me the way DCGAN would at the drop of a hat.
Have you found any good ways to speed it up? The five-fold training on the Critic is very expensive.
Deep Forest: Towards an Alternative to Deep Neural Networks
91–100 of 100 posts
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#92Looking forward to try the code (especially on CIFAR or ImageNet), Zhi-Hua Zhou, one of the authors, said they are going to publish it soon.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#93Earlier quoted context omitted.
Which is why you should do stereo or SfM, make a 3d reconstruction, and then do HOG or some 3D feature to recognise the coke can. Trying to do it from images with a NN that doesn't comprehend 3D space is just silly.
I'm not sure if you're serious or throwing some very excellent shade.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#94Earlier quoted context omitted.
Which is why you should do stereo or SfM, make a 3d reconstruction, and then do HOG or some 3D feature to recognise the coke can. Trying to do it from images with a NN that doesn't comprehend 3D space is just silly.
Errm, hang on, are you saying that if you have a task of classifying unseen images given a labelled training set you should get a stereo camera or video camera and create another problem? Which you can solve? Because the problem is silly> What if I say : "I will give you $10m to solve it, and if you fail, I will kill this very kind old monkey?"
Why, I'm not sure, but I'm guessing because it is hard/inaccurate to do with just NNs and parameter/network architecture tweaking. Possibly also because benchmarks with single mono images are much easier to make.
Just because it is hard with method A, and is harder to make benchmarks, doesn't mean method B isn't better.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#95Earlier quoted context omitted.
Errm, hang on, are you saying that if you have a task of classifying unseen images given a labelled training set you should get a stereo camera or video camera and create another problem? Which you can solve? Because the problem is silly> What if I say : "I will give you $10m to solve it, and if you fail, I will kill this very kind old monkey?"
Object recognition doesn't only exist in the subspace of labelled 2D images. It tends to be derived from a 3D space, which is a whole extra orthogonal data source that the "NN all the things" crowd is fastidiously ignoring. Why, I'm not sure, but I'm guessing because it is hard/inaccurate to do with just NNs and parameter/network architecture tweaking. Possibly also because benchmarks with single mono images are much…
Sure, if you are building a Robot and I say "use this camera and a deep network" and you say "It'll work better with stereo" well... yes super do that!
But if we are working with mono images I don't understand how the observation helps?
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#96"In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train." Hyperparameter tuning is not as much of an issue with deep neural networks anymore. Thanks to BatchNorm and more robust optimization algorithms, most of the time you can simply use Adam with a default learning rate of 0.001 and do pretty well. Dropout is not even necessary with many models tha…
No batch norm for LSTMs
EDIT: to clarify: "j/k" with the thing in parenthesis ;-)
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#97Earlier quoted context omitted.
Object recognition doesn't only exist in the subspace of labelled 2D images. It tends to be derived from a 3D space, which is a whole extra orthogonal data source that the "NN all the things" crowd is fastidiously ignoring. Why, I'm not sure, but I'm guessing because it is hard/inaccurate to do with just NNs and parameter/network architecture tweaking. Possibly also because benchmarks with single mono images are much…
Yes but am I missing something when I say that if the problem is to deal with labelled 2d images declaring that you should be working with 3d images or short video sequences doesn't help. Sure, if you are building a Robot and I say "use this camera and a deep network" and you say "It'll work better with stereo" well... yes super do that! But if we are working with mono images I don't understand how the observation he…
> If you want to recognize all Coke cans in all fridges, for your real-world, consumer-ready Coke-fetching robot product?
If you're stuck with a mono dataset, post collection, then sure use NN and call it a day. But even if you have video you can do 3D reconstruction just from baseline movement. You won't know scale, so you can't differentiate between big coke cans and little coke cans, but at least you can rule out pictures of coke cans.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#98Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#99Earlier quoted context omitted.
> If it did, humans wouldn't be able to choose a good algorithm for specific cases, and obviously we can. This is a surprisingly commonly held fallacy in some AI circles. It's the idea that humans are mathematically perfect. When you phrase it that way, it's fairly obviously false, but you still see a lot of people argue things like "NFL doesn't apply to ensembles because humans..." or "machines can never be as intel…
It doesn't imply that humans are mathematically perfect, but our brains are basically algorithm generating algorithms - we're not just weighting a set of preexisting solutions. To generalize quite a bit, saying NFL applies to general intelligence ends up implying there are problems for which efficient algorithms exist, but intelligence is literally incapable of discovering. That seems pretty absurd to me.
The second point is that NFL is often interpreted in a weird way, where we only think about "interesting problems". It is defined on the set of all search or optimization problems. It says if your algorithm is better than exhaustive search over some subset of problems, it must be worse on the complement of that set.
What does it mean for an algorithm to be efficient? Well, it's roughly speaking the number of steps it needed to take (assuming each step is the same amount of work, blah blah). OK, so an "efficient" algorithm must, by definition, prioritize some steps over others -- it's picking the "best" steps to take each time it has the choice. OK, so I'll just make up some instances of the problem that are custom-tailored so that what the algorithm thinks are the "best" steps always lead me in the wrong direction. You algorithm will then be worse than exhaustive search on my set of problems, precisely because it's choosing to avoid the steps I know to be good -- I defined the problem to make that happen.
That is true of any algorithm you can conceive of. There will be problems for which the bias that makes the algorithm good on the problems you intended it to work on will be exactly the wrong bias.
It doesn't matter if you say, "Aha, I'll let my algorithm generate new algorithms! Gotcha!" I'll just design a set of problems for which your algorithm generating algorithm will generate the wrong algorithms.
Search is always about bias. Without bias, you have random search -- that's literally the textbook machine learning definition of bias. All NFL says is that if you have to worry about every possible problem, any bias you choose will be worse than random sometimes. There's no escape clause here. Your algorithm generating algorithm is still a search algorithm with its own biases, and it will still be worse than random on some subset of all possible problems.
Re: Deep Forest: Towards an Alternative to Deep Neural Networks
#100No Free Lunch theorem refesher: "if an algorithm performs well on a certain class of problems then it necessarily pays for that with degraded performance on the set of all remaining problems" https://en.m.wikipedia.org/wiki/No_free_lunch_theorem
There is a theoretically optimal machine learning algorithm called Solomonoff Induction. Solomonoff Induction assumes a prior over all possible computer programs that could have produced the data. And it assumes that shorter, simpler programs are more likely than longer, more complex ones. Under such a prior, the no free lunch theorem doesn't apply. NFLT assumes that all machine learning problems have infinite inform…