Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

github.com

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#21

Post by jmward01 »

Etheryte wrote: post #18
steinvakt wrote: post #16

How does the accuracy compare to Whisper?

This uses SenseVoice under the hood, which claims to have better accuracy than Whisper. Not sure how accurate that statement is though, since I haven't seen a third party comparison, in this space it's very easy to toot your own horn. [0] https://github.com/FunAudioLLM/SenseVoice

This uses SenseVoice small under the hood. They claim their large model is better than Whisper large v3, not the small version. This small version is definitely worse than Whisper large v3 but still usable and the extra annotation it does is interesting.

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#24

Post by throwaway2016a »

This looks really nice. What I find interesting is that it seems to advertise itself for the transcription use case but if it is "lightning fast" I wonder if there are better uses cases for it.

I use AWS Transcribe[1] primarily. It costs me $0.024 per minute of video and also provides timestamps. It's unclear to me without running the numbers if using this model I could do any better than that seeing as it needs a GPU to run.

With that said, I always love to see these things in the Open Source domain. Competition drives innovation.

Edit: Doing some math, with spot instances on EC2 or serverless GPU on some other platforms it could be relatively price competitive with AWS Transcribe if the performance is even slightly fast (2 hours of transcription per hour to break even). Of course the devops work for running your own model is higher.

[1] https://aws.amazon.com/transcribe/

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#25

Post by ks2048 »

steinvakt wrote: post #16

How does the accuracy compare to Whisper?

I've been doing some things with Whisper and find the accuracy very good, BUT I've found the timestamps to be pretty bad. For example, using the timestamps directly to clip words or phrases often clips off the end of word (even simple cases where is followed by silence). Since this emphases word timestamps, I may give it a try.

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#26

Post by yellow_lead »

jbellis wrote: post #22

OOMs even in quantized mode on a 3090. What's a better option for personal use? > torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 43.71 GiB. GPU 0 has a total capacity of 24.00 GiB of which 20.74 GiB is free.

Not sure if you mean in general, or options for this particular project, but Whisper should work for you.

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#27

Post by ChrisMarshallNY »

throwaway2016a wrote: post #24

This looks really nice. What I find interesting is that it seems to advertise itself for the transcription use case but if it is "lightning fast" I wonder if there are better uses cases for it. I use AWS Transcribe[1] primarily. It costs me $0.024 per minute of video and also provides timestamps. It's unclear to me without running the numbers if using this model I could do any better than that seeing as it needs a GP…

> better uses cases for it.

I want my babelfish!

Re: Omni SenseVoice: High-Speed Speech Recognition with Words Timestamps

#28

Post by khimaros »

Etheryte wrote: post #18
steinvakt wrote: post #16

How does the accuracy compare to Whisper?

This uses SenseVoice under the hood, which claims to have better accuracy than Whisper. Not sure how accurate that statement is though, since I haven't seen a third party comparison, in this space it's very easy to toot your own horn. [0] https://github.com/FunAudioLLM/SenseVoice

this claims to have speaker diarization which is a potentially killer feature missing from most whisper implementations.
Post reply on HN