500 million parameters seems like a lot, are there not duplication or redundancies that can reduce the parameters.
One could also use batches of data.
Seems very expensive!
How to train large deep learning models as a startup
51–60 of 84 posts
Re: How to train large deep learning models as a startup
#52in my experience it's often more like "just use linear regression and tell everyone you're using AI"
the serious tip here is to go with gradient boosting which very often works so well it hardly makes a difference
Re: How to train large deep learning models as a startup
#53the hardest part here is horizontal scaling. OpenAI handrolled its own MPI+SSH stack (https://openai.com/blog/scaling-kubernetes-to-7500-nodes/)
I wonder what is the state of art for horizontal scaling here ...preferably on kubernetes.
Pytorch is tricky to integrate (using TorchElastic). You could use Dask or Ray Distributed. Tensorflow has its own mechanism that doesnt play nice with Kubernetes.
How are others doing it ?
Re: How to train large deep learning models as a startup
#54Earlier quoted context omitted.
Thats for structured data, for non structured it's more like "create a NN and stack more layers until you have your MVP"
> "create a NN and stack more layers until you have your MVP" I mean, that's a pretty good principled approach to a lot of ML problems.
Only because currently ML is more alchemy than engineering. We mix stuff until we make gold while we can't explain why more parameters generalize better instead of overfitting.
Re: How to train large deep learning models as a startup
#55Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.
I believe most people already moved to offline engines. No need to send the data to some random guys like this Assembly. Nemo Conformer from Nvidia, Robust Wav2Vec from Facebook, Vosk. There are dozen options. And the cost is $0.01 per hour, not $0.89 per hour like here. Another advantage is that you can do more custom things - add words to vocabulary, detect speakers with biometric features, detect emotions.
without talking about accuracy any comparison is meaningless.
Re: How to train large deep learning models as a startup
#56If you wanted to do something like "OK Google" with AssemblyAI would you have to transcribe everything and then process the substring "OK Google" on the application layer (and therefore incur all of the cost of listening constantly)? It'd be cool if there was the ability to train a phrase locally on your own premises and then use that to begin the real transcription. This probably wouldn't be super difficult to build…
Bose used to have some pre internet system that recognized the song you liked to play right after another song (like in a random shuffle) and attempted to learn what you liked to hear, and queue up the song you were likely to skip to anyway. No idea how they pulled it off since this must have been on hardware from 15 years ago iirc.
Ah yes Bose uMusic. From the manual it extracts 30 feature points from the songs to define your preference.
uMusic patent: https://patents.google.com/patent/CN1637743A/en
Further reading: http://products.bose.com/pdf/customer_service/owners/uMusic_...
Re: How to train large deep learning models as a startup
#57Earlier quoted context omitted.
I believe most people already moved to offline engines. No need to send the data to some random guys like this Assembly. Nemo Conformer from Nvidia, Robust Wav2Vec from Facebook, Vosk. There are dozen options. And the cost is $0.01 per hour, not $0.89 per hour like here. Another advantage is that you can do more custom things - add words to vocabulary, detect speakers with biometric features, detect emotions.
without talking about accuracy any comparison is meaningless.
You don't even need to compare accuracy, you can just check the technology. Facebook model is trained on 256 GPU cards and you can fine-tune it to your domain in a day or two. The release was 2 month ago. There is no way any cloud startup can have something better in production given they have access to just 4 Titan cards.
Re: How to train large deep learning models as a startup
#58> How to train large deep learning models as a startup
How to train large deep learning models at a well founded startup*
Everything described here is absolutely not affordable by bootstrappers and startups with little funding, unless the model to train is not that deep.
Re: How to train large deep learning models as a startup
#59> How to train large deep learning models as a startup How to train large deep learning models at a well founded startup* Everything described here is absolutely not affordable by bootstrappers and startups with little funding, unless the model to train is not that deep.
As a bootstrapper I camped all night outside of bestbuy to get some 3090s.
Other tips not mentioned in the article:
1. Tune your hyper parameters on a subset of the data.
2. Validate new methods with smaller models on public datasets.
3. Tune models instead of training from scratch (either public models or your previously trained ones).
Re: How to train large deep learning models as a startup
#60> How to train large deep learning models as a startup How to train large deep learning models at a well founded startup* Everything described here is absolutely not affordable by bootstrappers and startups with little funding, unless the model to train is not that deep.
As a bootstrapper I camped all night outside of bestbuy to get some 3090s. Other tips not mentioned in the article: 1. Tune your hyper parameters on a subset of the data. 2. Validate new methods with smaller models on public datasets. 3. Tune models instead of training from scratch (either public models or your previously trained ones).
Great hacks, although you have to be aware of the trade-offs:
1. if you choose the wrong subset, you'll find a non optimum local min
2. still risk dead ends when expanding the model and lengthen the time to finding that out
3. a lot of public models are made from inaccurate datasets, so beware
Overall you have to start somewhere though, and your points are still valid.