Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to train large deep learning models as a startup

assemblyai.com

Post reply on HN
Print view View original

21–30 of 84 posts

Re: How to train large deep learning models as a startup

#21

Post by 6gvONxR4sf7o »

visarga wrote: post #4

> that still adds up to $2,451,526.58 to run 1,024 A100 GPUs for 34 days Salary costs are probably even higher than compute costs. Automatic Speech Recognition is an industrial scale application, it costs a lot to train, but so do many other projects in different fields. How expensive is a plane or a ship? How much can a single building cost? A rocket launch?

In what way are salary costs higher? This is on the order of 10 of their people’s annual salaries. This is for a single training run (meaning overall compute costs are higher), and it isn’t the only thing those ten or so people would have done that year (also meaning overall compute costs are higher).

Re: How to train large deep learning models as a startup

#22

Post by singularity2001 »

freshthought wrote: post #2

Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.

Maybe relevant in context: you can now use Siri offline transcription inside your apps. (for free)

Re: How to train large deep learning models as a startup

#23

Post by 6gvONxR4sf7o »

freshthought wrote: post #2

Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.

This doesn’t answer the question at all, but huggingface also has some decent ASR models available.

Re: How to train large deep learning models as a startup

#24

Post by cleverpebble »

I definitely enjoyed reading your article!

Did you play around with any AI-specific accelerators (eg TPUs?)

Looking at some basic cost analysis from a stranger on the Internet - https://medium.com/bigdatarepublic/cost-comparison-of-deep-l... - you can probably get a decent price reduction in training, especially using preemptive instances (and perhaps a better pricing contract with Google/AWS)

It's kind of crazy how the shortage of GPUs is affecting pricing on physical devices. My RTX Titan I bought in 2019 for $2,499 runs almost $5k on Amazon and is in short supply. The Titan V options you linked (although I think theres a typo because you referred it it as a Titan X) is an option - but it is still super overpriced for it's performance. Of course, this will probably settle down in the next year or two, and by then there will be new GPUs that are ~2-4x flop/$ compared to the V100/A100.

Re: How to train large deep learning models as a startup

#25

Post by kettleballroll »

cleverpebble wrote: post #24

I definitely enjoyed reading your article! Did you play around with any AI-specific accelerators (eg TPUs?) Looking at some basic cost analysis from a stranger on the Internet - https://medium.com/bigdatarepublic/cost-comparison-of-deep-l... - you can probably get a decent price reduction in training, especially using preemptive instances (and perhaps a better pricing contract with Google/AWS) It's kind of crazy how…

At these sizes, tpu would definitely be the way to go, and would likely be a lot cheaper (and potentially faster) than GPUs.

Re: How to train large deep learning models as a startup

#27

Post by trowngon »

freshthought wrote: post #2

Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.

I believe most people already moved to offline engines. No need to send the data to some random guys like this Assembly. Nemo Conformer from Nvidia, Robust Wav2Vec from Facebook, Vosk. There are dozen options. And the cost is $0.01 per hour, not $0.89 per hour like here.

Another advantage is that you can do more custom things - add words to vocabulary, detect speakers with biometric features, detect emotions.

Re: How to train large deep learning models as a startup

#28

Post by Learnedvector »

cleverpebble wrote: post #24

I definitely enjoyed reading your article! Did you play around with any AI-specific accelerators (eg TPUs?) Looking at some basic cost analysis from a stranger on the Internet - https://medium.com/bigdatarepublic/cost-comparison-of-deep-l... - you can probably get a decent price reduction in training, especially using preemptive instances (and perhaps a better pricing contract with Google/AWS) It's kind of crazy how…

Last I've checked (a year or two ago) PyTorch support for TPU's were atrocious. Have they gotten any better?

Re: How to train large deep learning models as a startup

#29

Post by nshm »

6gvONxR4sf7o wrote: post #23
freshthought wrote: post #2

Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.

This doesn’t answer the question at all, but huggingface also has some decent ASR models available.

Huggingface ASR models are not really recommended. The simple fact they don't use beam decoder with LM makes them much less accurate for practical applications. If you compare them to setups like Nemo + pyctcdecode, they will be 30% less accurate.

Also, most of the models there are undertrained.

Post reply on HN