Population-based training of neural networks • Hacker News Board

Re: Population-based training of neural networks

#21

Post by jicks » Tue, Nov 28, 2017, 4:39 PM UTC

giacaglia wrote: post #10

This seems similar to what Jeff Dean was working on with AutoML: https://research.googleblog.com/2017/11/automl-for-large-sca... . Is DeepMind collaborating with the Google Brain team and how connected are the teams? It seems somehow that the efforts may be duplicated in some areas...

It is actually very different:

- AutoML is used to automate the design of the ML model.

- Population-based trained is used to automate the choice of the hyperparameters (e.g. the rate of learning).

If you wanted to use both, you'd first use AutoML to find a good design for your problem, and then you'd use PBT when training your network.

Re: Population-based training of neural networks

#22

Post by kirillseva » Tue, Nov 28, 2017, 4:55 PM UTC

nl wrote: post #20

taneq wrote: post #19

Earlier quoted context omitted.

Does Theano meet your needs? Then no. Does TensorFlow meet them better, enough to justify the cost in switching? Then yes. "Actively developed" is a silly metric. Focus on features, flexibility, robustness etc.

For neural network libraries this isn't sensible. For many (most?) users outside of Google and Facebook the most important feature is "is there an off-the-shelf implementation of new technique XXX or do I have to build it myself?" For most users the sensible choice comes down to Keras+Tensorflow or PyTorch.

Depends on what you do. If you're starting a new project picking Theano indeed isn't a very good choice due to the reasons you've mentioned. However, if you already have a stable piece of software that does what you want it to do then migration won't add much value and you could spend this time doing something more important, like improving documentation or having dinner with your family and friends.

However it's worth pointing out that theano's API is somewhat similar to tensorflow so migrating shouldn't be too hard and should be fairly easy to test

Re: Population-based training of neural networks

#23

Post by hmm_really » Mon, Dec 04, 2017, 6:52 PM UTC

Looks like a genetic algorithm to me, just framed it differently to normal (which is clever). The hyperparams being the DNA and a network being an environment in which they are executed against.