Population-based training of neural networks
deepmind.com
Population-based training of neural networks
1–10 of 23 posts
Re: Population-based training of neural networks
#2This is really cool! I haven't read through the real paper yet but it's very impressive that this method does not incur a significant performance cost. I had assumed that using a genetic-style algorithm would be costly since you would need to train a large number of networks individually, but treating all variations equally in terms of training time now seems naive. Distributing the training time using intelligent exploration and exploitation is an awesome idea to fix this.
Re: Population-based training of neural networks
#3This is really cool! I haven't read through the real paper yet but it's very impressive that this method does not incur a significant performance cost. I had assumed that using a genetic-style algorithm would be costly since you would need to train a large number of networks individually, but treating all variations equally in terms of training time now seems naive. Distributing the training time using intelligent ex…
This is a fairly well represented technique in bayesian hyperparameter optimization, where you train a meta-classifier that keeps track of the parameter space. Kind of like a manager model, if you will, that learns to intelligently optimize exploration vs exploitation so that a team of workers will arrive at the global optimum.
Back when Yahoo! was a real company they used a technique called "multi-armed bandits" to learn what ads to show. [1]
More recently, there's a number of off-the-shelf packages available that you can trivially integrate into your ML pipeline to optimize hyperparameters of your models, I'll include the links below.
[1] multi-armed bandits https://www.theregister.co.uk/2011/09/23/yahoo_core_personal...
[2] tree of parzen estimators https://jaberg.github.io/hyperopt/
[3] hyperband - what google uses in their internal ML toolkits AFAIK https://arxiv.org/pdf/1603.06560.pdf
[4] (shameless plug) gaussian process based hyperparameter optimization service https://github.com/avantoss/loop
Re: Population-based training of neural networks
#4 So basically SGD based NEAT?
Re: Population-based training of neural networks
#5I think this is what Google and the others aim for - no hand-tuning. You simply specify the problem (some function to optimize) and the data. Everything in a nice concise package running on Google Cloud using their custom software and hardware.
Re: Population-based training of neural networks
#6So basically SGD based NEAT?
NEAT just uses GA to generate a network topology and weights. From what I read, I think this is just a fancy way to parallelize searching for optimal hyperparameters.
Re: Population-based training of neural networks
#7So basically SGD based NEAT?
NEAT just uses GA to generate a network topology and weights. From what I read, I think this is just a fancy way to parallelize searching for optimal hyperparameters.
Well, okay, what I meant is that it's literally applying evolutionary algorithms for hyperparameter optimization. Calling it some souped up BS like "population based training" seems like Deepmind marketing is getting out of hand...
Re: Population-based training of neural networks
#8This is really cool! I haven't read through the real paper yet but it's very impressive that this method does not incur a significant performance cost. I had assumed that using a genetic-style algorithm would be costly since you would need to train a large number of networks individually, but treating all variations equally in terms of training time now seems naive. Distributing the training time using intelligent ex…
This is a fairly well represented technique in bayesian hyperparameter optimization, where you train a meta-classifier that keeps track of the parameter space. Kind of like a manager model, if you will, that learns to intelligently optimize exploration vs exploitation so that a team of workers will arrive at the global optimum. Back when Yahoo! was a real company they used a technique called "multi-armed bandits" to…
This isn't your standard MAB or GP hyperparameter optimization; those typically require you to train each NN to convergence before further exploration is done (ie each 'round' is training a NN). Skimming the paper, OP is closer to freeze-thaw or reversible backpropagation hyperparameter optimization, or Net2Net meta-RL: the hyperparameter optimization is monitoring the loss curve of each trained NN, switching between them based on promisingess like in freeze-thaw, but also switching hyperparameters on the fly and reusing the trained weights to avoid starting from scratch, Net2Net style. Each NN being trained is periodically updated to either clone & tweak a new hyperparameter set to continue training the current NN's parameters, or clone & tweak the best NN's parameters while keeping the old hyperparameters. (They only clone the full NN, so they can't do architecture search, but there's no reason they couldn't use Net2Net or other recent approaches which similarly recycle the trained weights to avoid the huge computational burden of training from scratch.)
Re: Population-based training of neural networks
#9So basically SGD based NEAT?
NEAT paper link I post a lot on HN: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf
Looks like like a "not a GA we promise" on hyper-parameter selection (which is cool if unnecessarily obtuse)
Re: Population-based training of neural networks
#10This seems similar to what Jeff Dean was working on with AutoML: https://research.googleblog.com/2017/11/automl-for-large-sca....
Is DeepMind collaborating with the Google Brain team and how connected are the teams? It seems somehow that the efforts may be duplicated in some areas...