Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ersatz - Deep neural networks in the cloud

ersatz1.com

Post reply on HN
Print view View original

21–30 of 35 posts

Re: Ersatz - Deep neural networks in the cloud

#21

Post by tmadar »

RyanZAG wrote: post #2

Little bit confusing on what this actually is. Is this 1) Cloud GPU computation where you upload some special model code that is run on the neural network? ie. your own code 2) Upload data and run some pre-specified models on it, such as in the example you have a '-d model=spanish_speech_recognizer' - in which case the offering is all about how many and how good your pre-defined models are. The two different use case…

Very well explained. Hope you don't mind that I pimped your comment on my personal blog http://www.blogoftravis.com/

Re: Ersatz - Deep neural networks in the cloud

#23

Post by dave_sullivan »

thechut wrote: post #22

The company I work for could be very interested in testing this service. I requested a beta invite and filled out the survey. Any idea when you might start letting people test it out / accept beta invites?

Yes, most likely Monday. Are you with IOTworks? If so, I've got your survey and will be sure you're on the list.

Although I should add that the response so far has been way beyond what we thought it would be (which is fantastic!), so it may take some time to get to everyone. The beta literally just opened this morning, and I'm not ready to open up the product before working with beta users to polish it up.

Re: Ersatz - Deep neural networks in the cloud

#24

Post by cynusx »

benmccann wrote: post #17

This seems like a very small market you're going after. It requires people to have good knowledge of deep neural networks (they have to choose the model, architecture, hidden units, multiplicative units, etc.) I think it would be more interesting and open things up to a wider audience if some of these parameters could be chosen for you.

machine learning in general is a domain for specialists, it won't most of the time if you don't know what you are doing. As for a small market, I disagree. there may be few people who understand it but these are the ones that are put in charge of trillions of rows of data to analyze too.

Re: Ersatz - Deep neural networks in the cloud

#25

Post by bravura »

Recommendation: Reach out to my colleague James Bergstra, and build out automatic hyperparameter selection. This will make your offering work off-the-shelf, which is what is necessary for it to see wider adoption.

Why? The real pain in the ass in training a deep network is the hyperparameter selection.

What is your learning rate? What is your noise level? What is your regularization parameter?

Choosing these values is a far bigger pain than almost everything else combined.

Doing a grid search is intractable. Random hyperparameter search is better. You can use a sophisticated strategy, like Bergstra et al have proposed.

Re: Ersatz - Deep neural networks in the cloud

#26

Post by pilooch »

bravura wrote: post #25

Recommendation: Reach out to my colleague James Bergstra, and build out automatic hyperparameter selection. This will make your offering work off-the-shelf, which is what is necessary for it to see wider adoption. Why? The real pain in the ass in training a deep network is the hyperparameter selection. What is your learning rate? What is your noise level? What is your regularization parameter? Choosing these values i…

Link to the paper: http://jmlr.csail.mit.edu/papers/v13/bergstra12a.html

Re: Ersatz - Deep neural networks in the cloud

#27

Post by deadairspace »

tlarkworthy wrote: post #15

OMG I have been waiting for something like this. Deep Belief Networks have been smashing machine learning records in jsut about every domain. The only problem was that they were annoyingly slow to converge, and hard to program/debug see Hinton's google code slides for more info on how powerful these things are:- http://www.youtube.com/watch?v=AyzOUbkUf3M (that's 2007, things are even spicier now)

That was a great talk, and an impressive demo of feature generation.

Re: Ersatz - Deep neural networks in the cloud

#28

Post by ninjin »

bravura wrote: post #25

Recommendation: Reach out to my colleague James Bergstra, and build out automatic hyperparameter selection. This will make your offering work off-the-shelf, which is what is necessary for it to see wider adoption. Why? The real pain in the ass in training a deep network is the hyperparameter selection. What is your learning rate? What is your noise level? What is your regularization parameter? Choosing these values i…

I agree that the hyper-parameter selection is a huge pain, personally though, I am more familiar with the work of Snoek et al. [1] from NIPS in December last year. He even distributes a neat Python package that will perform Bayesian optimisation combined with MCMC [2] so that even people, like me, that are not yet familiar with Gaussian Processes can deploy it easily.

[1]: http://arxiv.org/pdf/1206.2944v2

[2]: http://www.cs.toronto.edu/~jasper/software.html

Re: Ersatz - Deep neural networks in the cloud

#30

Post by hosh »

dave_sullivan wrote: post #4
RyanZAG wrote: post #2

Little bit confusing on what this actually is. Is this 1) Cloud GPU computation where you upload some special model code that is run on the neural network? ie. your own code 2) Upload data and run some pre-specified models on it, such as in the example you have a '-d model=spanish_speech_recognizer' - in which case the offering is all about how many and how good your pre-defined models are. The two different use case…

Sure, I see the confusion. So basically, you bring the data, pick the neural network architecture you want to use, and set its parameters. The model trains on the data you've given it using a GPU cluster (which still takes a while) 'spanish_speech_recognizer' is the name of the model you just trained, where 'MRNN' is the actual architecture (a multiplicative recurrent neural network as described in http://www.cs.toro…

Thanks for explaining it this way.
Post reply on HN