Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

github.com

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#12

Post by woodson »

staticautomatic wrote: post #3

I’ve been building a production app on top of ASR and find the range of models kind of bewildering compared to LLMs and video. The commercial offerings seem to be custom or built on top of Whisper or maybe nvidia canary/parakeet and then you have stuff like speechbrain that seems to run on top of lots of different open models for different tasks. Sometimes it’s genuinely hard to tell what’s a foundation model and wha…

There’s just not a single one-size-fits-all model/pipeline. You choose the right one for the job, depending on whether you need streaming (i.e., low latency; words output right when they’re spoken), run on device (e.g. phone) or server, what languages/dialects, conversational or more “produced” like a news broadcast or podcast, etc. Best way is to benchmark with data in your target domain.

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#14

Post by staticautomatic »

woodson wrote: post #12
staticautomatic wrote: post #3

I’ve been building a production app on top of ASR and find the range of models kind of bewildering compared to LLMs and video. The commercial offerings seem to be custom or built on top of Whisper or maybe nvidia canary/parakeet and then you have stuff like speechbrain that seems to run on top of lots of different open models for different tasks. Sometimes it’s genuinely hard to tell what’s a foundation model and wha…

There’s just not a single one-size-fits-all model/pipeline. You choose the right one for the job, depending on whether you need streaming (i.e., low latency; words output right when they’re spoken), run on device (e.g. phone) or server, what languages/dialects, conversational or more “produced” like a news broadcast or podcast, etc. Best way is to benchmark with data in your target domain.

Sure, you're just going to try lots of things and see what works best, but it's confusing to be comparing things at such different levels of abstraction where a lot of the time you don't even know what you're comparing and it's impossible to do apples-to-apples even on your own test data. If your need is "speaker identification", you're going to end up comparing commercial black boxes like Speechmatics (probably custom) vs commercial translucent boxes like Gladia (some custom blend of whisper + pyannote + etc) vs [asr_api]/[some_specific_sepformer_model]. Like, I can observe that products I know to be built on top of whisper don't seem to handle overlapping speaker diarization that well, but I don't actually have any way of knowing if that's got anything to do with whisper.

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#15

Post by threeseed »

mkl wrote: post #6
leetharris wrote: post #4

Earlier quoted context omitted.

We released a new SOTA ASR as open source just a couple of weeks ago. https://www.rev.com/blog/speech-to-text-technology/introduci... Take a look. We'll be open sourcing more models very soon!

> These models are accessible under a non-commercial license. That is not open source.

Exactly. It is source available but not open source:

https://opensource.org/osd

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#18

Post by Etheryte »

steinvakt wrote: post #16

How does the accuracy compare to Whisper?

This uses SenseVoice under the hood, which claims to have better accuracy than Whisper. Not sure how accurate that statement is though, since I haven't seen a third party comparison, in this space it's very easy to toot your own horn.

[0] https://github.com/FunAudioLLM/SenseVoice

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#19

Post by pferdone »

Etheryte wrote: post #18
steinvakt wrote: post #16

How does the accuracy compare to Whisper?

This uses SenseVoice under the hood, which claims to have better accuracy than Whisper. Not sure how accurate that statement is though, since I haven't seen a third party comparison, in this space it's very easy to toot your own horn. [0] https://github.com/FunAudioLLM/SenseVoice

I mean they make a bold statement up top just to paddle back a little bit further down with: "[…] In terms of Chinese and Cantonese recognition, the SenseVoice-Small model has advantages."

It feels dishonest to me.

[0] https://github.com/FunAudioLLM/SenseVoice?tab=readme-ov-file...

Post reply on HN