Ersatz - Deep neural networks in the cloud • Hacker News Board

Re: Ersatz - Deep neural networks in the cloud

#31

Post by hosh » Thu, Jan 17, 2013, 11:33 PM UTC

benmccann wrote: post #17

This seems like a very small market you're going after. It requires people to have good knowledge of deep neural networks (they have to choose the model, architecture, hidden units, multiplicative units, etc.) I think it would be more interesting and open things up to a wider audience if some of these parameters could be chosen for you.

This is true. What is also true is that the neural nets themselves have much broader market.

Re: Ersatz - Deep neural networks in the cloud

#32

Post by indrax » Fri, Jan 18, 2013, 6:06 AM UTC

Can we export the trained model back out and use it, or only query it with the API?

Re: Ersatz - Deep neural networks in the cloud

#33

Post by tlarkworthy » Fri, Jan 18, 2013, 8:12 AM UTC

bravura wrote: post #25

Recommendation: Reach out to my colleague James Bergstra, and build out automatic hyperparameter selection. This will make your offering work off-the-shelf, which is what is necessary for it to see wider adoption. Why? The real pain in the ass in training a deep network is the hyperparameter selection. What is your learning rate? What is your noise level? What is your regularization parameter? Choosing these values i…

Presumably if you have a GPU backed cloud DBN. Hyper parameter selection is faster than one param per day. Also how to you choose the parameters to the hyper parameter tuner? I am never convinced these things work given no free lunch theorem.

Re: Ersatz - Deep neural networks in the cloud

#34

Post by mimog » Fri, Jan 18, 2013, 9:37 AM UTC

Isn't the CPU/GPU intensive part of working with neural networks the training of them? Once the network is trained to within an acceptable error-rate why would you need the cloud?

Re: Ersatz - Deep neural networks in the cloud

#35

Post by dave_sullivan » Sat, Jan 19, 2013, 7:08 PM UTC

bravura wrote: post #25

Recommendation: Reach out to my colleague James Bergstra, and build out automatic hyperparameter selection. This will make your offering work off-the-shelf, which is what is necessary for it to see wider adoption. Why? The real pain in the ass in training a deep network is the hyperparameter selection. What is your learning rate? What is your noise level? What is your regularization parameter? Choosing these values i…

Yeah, it's a really good point.

I haven't played with automatic parameter selection much (but have been seeing more papers on it recently) so I hadn't really considered it all that closely.

While I'd like to give people a fair amount of control over model parameters if they want, it probably is very important that I make things as turnkey as I can. Shouldn't be too tough to hack something together and make it an option during training.

While I'm trying to start things off relatively simply, the overall goal really is towards allowing people to create models that act as parts of much larger systems, maybe larger neural nets themselves. A sort of genetic algorithm that spawns new neural networks with random parameters and random connections to previous networks could be kind of neat, and making the base elements of those types of architectures (a single fully connected deep net, for example) easily accessible is a first step towards that goal.