Best Speech-to-text API with speaker diarization?

#1

Post by _nickanthony » Sun, May 05, 2024, 10:25 PM UTC

Whisper does not offer speaker diarization so we're looking to migrate to another API. We're having trouble finding the same quality as Whisper but with speaker diarization.

So far, we tried Google's Speech-to-text and Azure's speech to text, but both are less accurate and struggle with custom phrases.

We're working with large audio files (>30 minutes, >25 MB), but we can change how we parse up the files and the file types. All of our files are in Google Cloud Storage right now.

Curious if anyone has recommendations?

Re: Best Speech-to-text API with speaker diarization?

#2

Post by verdverm » Sun, May 05, 2024, 11:35 PM UTC

What if you combine the two, use Whisper for accuracy and Google for diarization?

Re: Best Speech-to-text API with speaker diarization?

#3

Post by mavelikara » Mon, May 06, 2024, 5:01 AM UTC

I have found Deepgram to be good.

Re: Best Speech-to-text API with speaker diarization?

#4

Post by jilijeanlouis » Mon, May 06, 2024, 6:25 AM UTC

Our API, Gladia, supports speaker diarization. We use a hybrid enterprise-grade ASR system for speech-to-text, with our own version of Whisper at its core, and state-of-the-art open source models for diarization. We process large audio files, and use a proprietary algorithm so that our users are not billed extra for duplicate audio channels, as many other providers do. Hope this helps. There's a free trial if you'd like to test, and here's our blog with more info: https://www.gladia.io/blog/gladia-speech-to-text-api-speaker...

Re: Best Speech-to-text API with speaker diarization?

#5

Post by java_beyb » Mon, May 06, 2024, 3:07 PM UTC

i haven't them recently but in the past I found smaller companies are much better at customization.

try assemblyai, deepgram, picovoice or speechmatics. picovoice is on-device, you gotta fine-tune the model, but it's pretty easy as it gives you pronunciation recommendations, and you can run them serverless. https://picovoice.ai/docs/leopard/#add-custom-vocabulary the others do it through an API call and you gotta find your own pronunciation: https://docs.speechmatics.com/features/custom-dictionary

if you wanna go with whisper you can use picoVoice falcon or pyannote for speaker diarization: https://picovoice.ai/blog/falcon-whisper-integration/ https://github.com/yinruiqing/pyannote-whisper