Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to train large deep learning models as a startup

assemblyai.com

Post reply on HN
Print view View original

31–40 of 84 posts

Re: How to train large deep learning models as a startup

#31

Post by subpar »

rememberlenny wrote: post #12
freshthought wrote: post #2

Does anyone use this? How does AssemblyAI compare to Google’s? We are considering adding speech recognition to a small part of our product.

I would strongly advise against using Google's ML apis. First, at my company Milk Video, we are huge fans of Assembly AI. The quality, speed and cost of their transcription is galaxies beyond the competition. Having worked in machine learning focused companies for a few years, I have been researching this exact question. I'm curious how I can better forecast the amount of ML talent I should expect to build into our t…

How good is their speaker labeling? We've been using the Google API but their diarization has been basically unusable for our application (transcripts of group conversations).

Re: How to train large deep learning models as a startup

#32

Post by jszymborski »

cardosof wrote: post #30
stayfrosty420 wrote: post #26

in my experience it's often more like "just use linear regression and tell everyone you're using AI"

Thats for structured data, for non structured it's more like "create a NN and stack more layers until you have your MVP"

> "create a NN and stack more layers until you have your MVP"

I mean, that's a pretty good principled approach to a lot of ML problems.

Re: How to train large deep learning models as a startup

#33

Post by debbiedowner »

endisneigh wrote: post #17

If you wanted to do something like "OK Google" with AssemblyAI would you have to transcribe everything and then process the substring "OK Google" on the application layer (and therefore incur all of the cost of listening constantly)? It'd be cool if there was the ability to train a phrase locally on your own premises and then use that to begin the real transcription. This probably wouldn't be super difficult to build…

This is actually a much simpler task than ASR and you can even easily train on a normal CPU even.

The best do it yourself instructions are in a book called Tiny ML.

Compared to super deep transformers, you'll find that deployed WW detectors are as simple as SVMs or 2 layer NNs.

Re: How to train large deep learning models as a startup

#34

Post by dylanbfox »

subpar wrote: post #31
rememberlenny wrote: post #12

Earlier quoted context omitted.

I would strongly advise against using Google's ML apis. First, at my company Milk Video, we are huge fans of Assembly AI. The quality, speed and cost of their transcription is galaxies beyond the competition. Having worked in machine learning focused companies for a few years, I have been researching this exact question. I'm curious how I can better forecast the amount of ML talent I should expect to build into our t…

How good is their speaker labeling? We've been using the Google API but their diarization has been basically unusable for our application (transcripts of group conversations).

Dylan from Assembly here. If you want to send me one of your audio files (my email is in my profile) I'd be happy to send you back the diarized results from our API.

You can also signup for a free account and test from the dashboard without having to write any code if that's easier.

Other than lots of crosstalk in your group conversations - is there anything else challenging about your audio (eg, distance from microphones, background noise, etc?)

Re: How to train large deep learning models as a startup

#35

Post by hedgehog »

One thing to note on the "Train with lower precision" is on newer hardware with TF32 support that gives you much of the speedup of FP16 without being as finicky. Doesn't save memory, but still useful. Automatic in PyTorch, not sure in TensorFlow:

https://pytorch.org/docs/stable/notes/cuda.html#tensorfloat-...

This is mostly important because these settings can significantly affect the price/perf evaluation for your specific model & the available hardware.

Re: How to train large deep learning models as a startup

#36

Post by asdff »

endisneigh wrote: post #17

If you wanted to do something like "OK Google" with AssemblyAI would you have to transcribe everything and then process the substring "OK Google" on the application layer (and therefore incur all of the cost of listening constantly)? It'd be cool if there was the ability to train a phrase locally on your own premises and then use that to begin the real transcription. This probably wouldn't be super difficult to build…

Bose used to have some pre internet system that recognized the song you liked to play right after another song (like in a random shuffle) and attempted to learn what you liked to hear, and queue up the song you were likely to skip to anyway. No idea how they pulled it off since this must have been on hardware from 15 years ago iirc.

Re: How to train large deep learning models as a startup

#37

Post by potatoman22 »

stayfrosty420 wrote: post #26

in my experience it's often more like "just use linear regression and tell everyone you're using AI"

Lol, very true haha. In actuality, I don't think most NN's are any more 'AI' than simpler models. The definition for AI is fleeting, though.

Re: How to train large deep learning models as a startup

#38

Post by elmomle »

Excellent and informative article--and a good bit of brand-building, I might say :-). One thing I'd love to see more writing about is prototyping and iterative development in these contexts--deep NNs are notoriously hard to get "right", and there seems to be a constant tension between model architecting, tuning hyperparameters, etc.--for example, you presumably don't want to have to wait a couple of weeks (and burn through thousands of dollars) seeing if one choice of hyperparameters works well for your chosen architecture.

Of course, some development practices, such as ensuring that your loss function works in a basic sense, are covered in many places. But I'd love to see more in-depth coverage of architecture development & development best practices. Does anyone know of any particularly good resources / discussions there?

Re: How to train large deep learning models as a startup

#39

Post by r-zip »

jszymborski wrote: post #32
cardosof wrote: post #30

Earlier quoted context omitted.

Thats for structured data, for non structured it's more like "create a NN and stack more layers until you have your MVP"

> "create a NN and stack more layers until you have your MVP" I mean, that's a pretty good principled approach to a lot of ML problems.

I think you have a different definition of "principled" from most people.

Re: How to train large deep learning models as a startup

#40

Post by andyxor »

cardosof wrote: post #30
stayfrosty420 wrote: post #26

in my experience it's often more like "just use linear regression and tell everyone you're using AI"

Thats for structured data, for non structured it's more like "create a NN and stack more layers until you have your MVP"

it's more like try different off-the-shelf models on some sample of data until the performance is somewhat acceptable

Unless you're Google, who even trains models from scratch these days, at most you do some fine-tuning

Post reply on HN