Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Keras 2

blog.keras.io

Post reply on HN
Print view View original

31–40 of 70 posts

Re: Introducing Keras 2

#31

Post by nl »

krick wrote: post #22

I'm only starting with all that machine-learning, NN stuff and as many others I want to ask for some guidance/resources/learning material. What I feel especially lacking is something very broad and generic, some overview of existing techniques (but not as naïve as Ng's ML course, I assume). There exist a lot of estimators and classifiers, there exist a lot of techniques and tricks to train models, there exist a lot o…

So how, for instance, do I even decide, that Random Forest is not enough for this task and I want to build some specific kind of neural net?

The problem here is that it's really hard to give generic advice. As an analogy this is like asking "how do I know if Rails is enough for this task".

The answer is usually "yes", but the specifics matter a lot.

So in this specific case (and I realize you aren't looking for specific advice here, but I think the principles are useful):

Random Forests are very powerful, and work really well for hundreds, maybe thousands of features, on large but not huge amounts of data and are fairly easy to train.

There are a large number of types of neural networks. One of the big advantages of deep neural networks is that that can reduce the need for manual feature engineering. For examples conventional neural networks extract features from images that work better than any human engineered features, and LSTMs (and variations) work well at extracting features from text. The problem with deep neural networks is that they (generally) need a lot of data to train.

So, as usual the answer is "it depends".

In industry though, 90% of the time the question isn't "what classifier should I use". It's "how do I get the data"/"how do I extract features" and then "lets try all the classifiers and see what works best".

Re: Introducing Keras 2

#32

Post by fchollet »

backpropaganda wrote: post #28

1. Still no support for multiple losses. Models like VAEs cannot be idiomatically implemented. The second loss has to be 'hacked' in. Notice how in the official example for VAE, the kl_loss is computed using variables which are NOT available via the loss function ( https://github.com/fchollet/keras/blob/master/examples/varia... ) 2. It's still an input->output paradigm, rather than a {input, output}->loss paradigm wh…

See the release notes: https://github.com/fchollet/keras/wiki/Keras-2.0-release-not...

- You can use a Keras model to compute some tensor(s), turn that into a loss, and manually add that loss to the model via `add_loss` (it just needs to only depend on the model's inputs).

- Not all of your model outputs have to have a loss associated with them. So you can do both {input, output}->loss and input->output in your workflow, as you wish. Effectively, losses and outputs are decoupled.

The VAE example hasn't yet been updated to use the `add_loss` feature, but it should be.

Re: Introducing Keras 2

#33

Post by gnipgnip »

AlexCoventry wrote: post #19
gnipgnip wrote: post #16

Earlier quoted context omitted.

The only "math" in deep learning is given by reverse mode AD (or if you're into fancy stuff, "efficient computing of pullbacks"). The rest of it plain old hacking, and empirical tricks with the occasional variational doodads.

You clearly haven't read the book.

Don't need to when you read papers.

Re: Introducing Keras 2

#34

Post by dave_sullivan »

krick wrote: post #29
dave_sullivan wrote: post #26

Earlier quoted context omitted.

I'd definitely watch the first few episodes of Ng's stuff, up to and including logistic regression (unless you know all of that already, in which case: read papers and do practice projects for yourself--or compete in kaggle if you don't have any application ideas) The most common way to apply machine learning is supervised classification. The basic formula is: we learn a model (set of weights) to approximately map da…

It surely does make sense to me, but I seriously think (maybe hope, even?) that "hacking-driven" approach here is significantly overvalued. Because of sociological reasons. After all, all this is mathematical problems, and while I'm aware that NNs are pretty much unexplored space, there surely must exist some quite significant amount of knowledge at level below the NNs that can be actually systematically learned . Al…

If you just pick up a math book, you'll learn lots of stuff that you don't need to know. That's fine, but it strikes me as a good way to avoid actually doing anything and gaining practical experience.

If you hit a wall in practice because you don't understand the math, you'll usually have enough of an idea of the problem to ask more intelligent questions about what kind of math you need. That will, incidentally, help you understand the math better because you're coming to it out of an actual need rather than just seeing it mixed into a bunch of chapters.

Unless you're going to write a machine learning framework or be a researcher, the required math isn't too bad and it sounds like you might have enough of a background already. So don't be afraid to dive into something practical (like a kaggle competition).

FWIW this is a really good blog for insight into the math and intuition behind deep learning: http://colah.github.io/ (i'm not sure if it's quite what you're looking for though)

Re: Introducing Keras 2

#35

Post by mattkrause »

nl wrote: post #31
krick wrote: post #22

I'm only starting with all that machine-learning, NN stuff and as many others I want to ask for some guidance/resources/learning material. What I feel especially lacking is something very broad and generic, some overview of existing techniques (but not as naïve as Ng's ML course, I assume). There exist a lot of estimators and classifiers, there exist a lot of techniques and tricks to train models, there exist a lot o…

So how, for instance, do I even decide, that Random Forest is not enough for this task and I want to build some specific kind of neural net? The problem here is that it's really hard to give generic advice. As an analogy this is like asking "how do I know if Rails is enough for this task". The answer is usually "yes", but the specifics matter a lot. So in this specific case (and I realize you aren't looking for speci…

"Try 'em all" is not just an answer, but the only answer.

The No Free Lunch Theorem says that averaged across all possible problems, no single classifier is the best; in fact, they're all equivalent.

However, you probably don't care about all possible problems, but a specific one. Over the last decade or so, we've discovered that deep learning works really well on certain classes of problems, particularly those that may have some kind of nested structure, as in object or speech recognition. If your problem resembles one of those, a deep neural network might be a good place to start.

Re: Introducing Keras 2

#36

Post by jph00 »

krick wrote: post #22

I'm only starting with all that machine-learning, NN stuff and as many others I want to ask for some guidance/resources/learning material. What I feel especially lacking is something very broad and generic, some overview of existing techniques (but not as naïve as Ng's ML course, I assume). There exist a lot of estimators and classifiers, there exist a lot of techniques and tricks to train models, there exist a lot o…

Generally for structured data (i.e. each column represents a distinct type of information, such as 'revenue' or 'color') you'll want random forest or GBM.

For unstructured data, where you'll need lots of complex feature engineering, you'll generally want to let the model learning those features - so use deep learning. E.g. images, natural language, audio...

I've won competitions with random forests and teach deep learning - both definitely have their place, but they are generally for quite different types of data. (This may change in the future, however, with deep learning showing that it has the potential to work well for structured data too.)

(Don't worry about the No Free Lunch theorem - it has little to do with predictive modeling in the real world. Recent research shows that a random forest will give amongst the best results for the vast majority of real world datasets.)

Re: Introducing Keras 2

#37

Post by chas »

jph00 wrote: post #36
krick wrote: post #22

I'm only starting with all that machine-learning, NN stuff and as many others I want to ask for some guidance/resources/learning material. What I feel especially lacking is something very broad and generic, some overview of existing techniques (but not as naïve as Ng's ML course, I assume). There exist a lot of estimators and classifiers, there exist a lot of techniques and tricks to train models, there exist a lot o…

Generally for structured data (i.e. each column represents a distinct type of information, such as 'revenue' or 'color') you'll want random forest or GBM. For unstructured data, where you'll need lots of complex feature engineering, you'll generally want to let the model learning those features - so use deep learning. E.g. images, natural language, audio... I've won competitions with random forests and teach deep lea…

Which research are you referring to?

Re: Introducing Keras 2

#38

Post by apaszke »

jph00 wrote: post #14
fchollet wrote: post #12

Earlier quoted context omitted.

With the functional API of Keras, it would definitely make sense. In fact I do think that imperative model definition would be great to have at some point in the future. We'll see :)

I'm intrigued!... The kernel calling overhead and lack of any GPU while/scan/map/etc for Pytorch seems like a limitation, but I guess on 2nd thoughts you can still do all the keras fit/predict stuff and auto-connecting up the layers.

These ops are just not needed in PyTorch. while is just a Python while loop. Scan is a for loop, map is a list comprehension that applies modules. No need for anything fancy.

Re: Introducing Keras 2

#39

Post by Kiro »

Is it better to learn Keras instead of tflearn?

Copying a comment I made in another thread where one response recommended Keras:

I currently have a small pet project where I think some simple ML would be cool but I don't know where to start.

Basically my use case is that I have a bunch of 64x64 images (16 colors) which I manually label as "good", "neutral" or "bad". I want to input this dataset and train the network to categorize new 64x64 images of the same type.

The closest I've found is this: https://gist.github.com/sono-bfio/89a91da65a12175fb1169240cd...

But it's still too hard to understand exactly how I can create my own dataset and how to set it up efficiently (the example is using 32x32 but I also want to factor in that it's only 16 colors; will that give it some performance advantages?).

Re: Introducing Keras 2

#40

Post by HelloNurse »

evernflow wrote: post #25
juxtaposicion wrote: post #5

Will Keras2 support PyTorch as backend, in the future? Answer: [0] No, there are no plans to support PyTorch. There is nothing to be gained in supporting every novelty framework that crops up every quarter. Our goal is to make deep learning accessible and useful to as many people as possible, and that goal is completely opposite to building up deep learning hipster cred. [0]: https://github.com/fchollet/keras/issues/…

This is surprising since keras basically started as a rip off of the Torch API in Python

Layering Keras on top of another framework, such as Theano, is useful because it gains compatibility with code using that other framework. If Keras and PyTorch are both similar (in spirit and API) to Torch, integrating PyTorch-based code as is into Keras project would be very low-value compared to a presumably easy translation to Keras.
Post reply on HN