How to train large deep learning models as a startup
assemblyai.com
How to train large deep learning models as a startup
1–10 of 84 posts
Re: How to train large deep learning models as a startup
#2Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.
Re: How to train large deep learning models as a startup
#3[deleted]
Re: How to train large deep learning models as a startup
#4> that still adds up to $2,451,526.58 to run 1,024 A100 GPUs for 34 days
Salary costs are probably even higher than compute costs. Automatic Speech Recognition is an industrial scale application, it costs a lot to train, but so do many other projects in different fields. How expensive is a plane or a ship? How much can a single building cost? A rocket launch?
Re: How to train large deep learning models as a startup
#5Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.
Dylan from Assembly here. Most of our customers have actually switched over to us from Google - this Launch HN from a YC startup that uses our API goes into a bit more detail if you're interested:
https://news.ycombinator.com/item?id=26251322
My email is in my profile if you want to reach out to chat more!
Re: How to train large deep learning models as a startup
#6Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.
Also curious, are there any 'independent' performance benchmarks in this space?
Re: How to train large deep learning models as a startup
#7> that still adds up to $2,451,526.58 to run 1,024 A100 GPUs for 34 days Salary costs are probably even higher than compute costs. Automatic Speech Recognition is an industrial scale application, it costs a lot to train, but so do many other projects in different fields. How expensive is a plane or a ship? How much can a single building cost? A rocket launch?
> Salary costs are probably even higher than compute costs.
Yes exactly. Managing that much compute requires many humans!
Re: How to train large deep learning models as a startup
#8Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.
Also curious, are there any 'independent' performance benchmarks in this space?
This is tricky. The de facto metric to evaluate an ASR model is Word Error Rate (WER). But results can vary widely depending on the pre-processing that's done (or not done) to transcription text before calculating a WER.
For example if you take the WER of "I live in New York" and "i live in new york" the WER would be 60% because you're comparing a capitalized version vs an uncapitalized version.
This is why public WER results vary so widely.
We publish our own WER results and normalize the human and automatic transcription text as much as possible to get as close to "true" numbers as possible. But in reality, we see a lot of people comparing ASR services simply by doing diffs of transcripts.
Re: How to train large deep learning models as a startup
#9Check out Determined https://github.com/determined-ai/determined to help manage this kind of work at scale: Determined leverages Horovod under the hood, automatically manages cloud resources and can get you up on spot instances, T4's, etc. and will work on your local cluster as well. Gives you additional features like experiment management, scheduling, profiling, model registry, advanced hyperparameter tuning, etc.
Full disclosure: I'm a founder of the project.
Re: How to train large deep learning models as a startup
#10Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.
I used both Google's speech-to-text APIs and Assembly's APIs as well as some other ones to build Twilio Voice phone calling applications. The out of the box accuracy was way better with Assembly and its far easier to quickly customize the language model for higher accuracy in specific domains (for example programming language keywords). Generally I avoid using Google APIs whenever possible since they always seem overly complicated to get started with and have incomplete documentation even when I'm working in Python which should be one of the better supported languages.