Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

291–300 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#291

Post by thfuran »

guelo wrote: post #248
wging wrote: post #127

Earlier quoted context omitted.

Doesn't it mean 100% of your content needs to be double-checked? You can't easily identify which 2-3% of your content has errors. I'm aware that errors are more likely when the model is less confident of its predictions, but that shouldn't be enough. (edit for clarification: errors are not always something like "[UNINTELLIGIBLE]", where the system knows it doesn't know; they can also be misrecognitions that the syste…

Maybe you could run the text through a grammar checker to identify the errors.

That might work if people were required to speak grammatically.

Re: Whisper – open source speech recognition by OpenAI

#292

Post by lunixbochs »

coder543 wrote: post #261
lunixbochs wrote: post #238

Earlier quoted context omitted.

Ok, my test harness is ready. My A40 box will be busy until later tonight, but on an NVIDIA A2 [1], this is the batchsize=1 throughput I'm seeing. Common Voice, default Whisper settings, card is staying at 97-100% utilization: tiny.en: ~18 sec/sec base.en: ~14 sec/sec small.en: ~6 sec sec/sec medium.en: ~2.2 sec/sec large: ~1.0 sec/sec (fairly wide variance when ramping up as this is slow to process individual clips)…

Isn’t the A2 much weaker than a 3090? So those results are promising. EDIT: for what it's worth, Nvidia rated the A2 at 18 TFLOPS of FP16, and Apple rates the current A16 Neural Engine at 17 TFLOPS of FP16. I'm sure it's not an "apples to apples" comparison.

If you count the GPU component and memory bandwidth, the Apple M2 is slightly weaker on paper for 16-bit inference than the NVIDIA A2, if you manage to use the whole chip efficiently. The A16 is then slightly weaker than the M2.

Sure, the Whisper Tiny model is probably going to be fast enough, but from my preliminary results I'm not sure it will be any better than other models that are much much faster at this power class.

Whisper Large looks pretty cool, but it seems much harder to run in any meaningful realtime fashion. It's likely pretty useful for batch transcription though.

Even if you hit a realtime factor of 1x, the model can leverage up to 30 seconds of future audio context. So at 1x, if you speak for 10 seconds, you'll potentially need to wait another 10 seconds to use the result. This kind of latency is generally unsatisfying.

Re: Whisper – open source speech recognition by OpenAI

#293

Post by aidenn0 »

aidenn0 wrote: post #197
aidenn0 wrote: post #122

I just threw a random rock MP3 at it, and a first readthrough shows no transcription errors; this is quite good. Now I just want OCR that's even 50% as good as this...

Ran a few other songs through it and found one obvious mistranscription: "He's the bedroom cosmic rocker" (should be "He's the veteran cosmic rocker" in Veteran Cosmic Rocker by The Moody Blues) I also noticed that it's a little on the conservative side for detecting speech; all songs were missing at least part of one line.

Ran it on Juicy by The Notorious B.I.G and results were considerably worse than my mix of prog-rock and british invasion music I had tried before, though at least some of that is due to the number of proper-nouns in that song.

It took about 1000 CPU-minutes for this 5 minute song on my Ryzen 2700 with 12 OpenMP threads (about 100 minutes wall-clock).

Re: Whisper – open source speech recognition by OpenAI

#294

Post by aidenn0 »

magicalhippo wrote: post #279
aidenn0 wrote: post #234

Earlier quoted context omitted.

This should, in theory, work with CUDA; my GPU doesn't have enough RAM to do it (it runs out at 2.9GiB allocated, I have 4GiB, but am running a compositing desktop, which chews up about 600MiB; not sure where the other ~400MiB went) [edit] I confirmed CUDA worked with the "small" model, which used 3.3GB of GPU ram, and resulted in much poorer recognition than the "medium" model on my CPU (but it ran at least two orde…

CUDA worked fine with large on my 2080Ti FWIW. The speedup is ridiculous, as expected. My Ryzen 3800X used almost an hour transcribing a minute worth of speech, while the 2080Ti does it in like 10-20 seconds.

How much GPU ram did it use?

Re: Whisper – open source speech recognition by OpenAI

#295

Post by notahacker »

thfuran wrote: post #137
anigbrowl wrote: post #116

Earlier quoted context omitted.

No it isn't. That just means 2-3% of your content needs to be double-checked by a person at the audio level, saving huge amounts of time - equally true of human transcription, in which individual words are often [UNINTELLIGEBLE]. Would you want to review this fully before going into court, absolutely - because you'd want to play the recording to a jury for emotional impact. Can you rely on it when you want to quickly…

>equally true of human transcription, in which individual words are often [UNINTELLIGEBLE]. ML systems somewhat notoriously do not necessarily make the same sorts of errors that a human would. And I'd expect a large portion of the errors to be transcribing the wrong words rather that indicating that a word couldn't be transcribed. That sort of error means that you can't really get away with manually reviewing just 3%…

ML tending to make weird mistakes rather than subtle ones that make sense in context like human transcribers is likely to make them easier to spot.

And there are humans in the loop too, and an enormous amount of redundancy in the questions and answer, so even plausible false transcriptions will get picked up on if they matter. Nobody gets sent to jail simply because the transcription process - human or machine - accidentally substitutes "I did it" in place of "I didn't" midway through a two hour interview.

Re: Whisper – open source speech recognition by OpenAI

#297

Post by lunixbochs »

MacsHeadroom wrote: post #186
dom96 wrote: post #163

This really makes me want to build a Amazon Echo/Google Nest/etc replacement that's open hardware, open source and most importantly recognises voice completely offline. I find that I don't use these smart devices for much more than setting timers anyway so this seems like an easy project. I just wonder what system requirements Whisper has and whether there are open source voice recognition models that are specificall…

I really want all this too. The smallest model is ~80mb and the largest is 3gb. Not sure about system requirements yet; but models that small suggest this may be doable locally on a single board computer. Edit: According to this comment[0] the base model runs in real time on an M1 CPU. The tiny model apparently decodes an audio file twice as fast. These are promising results. [0] https://news.ycombinator.com/item?id=…

For an offline (non-streaming) model, 1x realtime is actually kind of bad, because you need to wait for the audio to be available before you can start processing it. So if you wait 10 seconds for someone to finish speaking, you won't have the result until 10 seconds after that.

You could use really small chunk sizes and process them in a streaming fashion, but that would impact accuracy, as you're significantly limiting available context.

Re: Whisper – open source speech recognition by OpenAI

#298

Post by mvexel »

hijp wrote: post #130
dceddia wrote: post #103

Earlier quoted context omitted.

Yep, I had this too. `pip3 install -U pip setuptools` took care of it. (If you get an error about pip3, try `pip` instead)

I'm really new to pip, but does this look ok? (after running the command for setuptools) Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: pip in /Users/xxx/Library/Python/3.9/lib/python/site-packages (22.2.2) Requirement already satisfied: setuptools in /Users/xxx/Library/Python/3.9/lib/python/site-packages (65.3.0) ---- after trying whisper installation: ×…

Not quite sure if this is related, but since there's a bunch of statements in there referencing rust: I had to install the rust compiler on my Mac (`brew install rust` if you use homebrew). This is not mentioned in the installation instructions.

Re: Whisper – open source speech recognition by OpenAI

#299

Post by magicalhippo »

aidenn0 wrote: post #294
magicalhippo wrote: post #279

Earlier quoted context omitted.

CUDA worked fine with large on my 2080Ti FWIW. The speedup is ridiculous, as expected. My Ryzen 3800X used almost an hour transcribing a minute worth of speech, while the 2080Ti does it in like 10-20 seconds.

How much GPU ram did it use?

I'm on Windows, using Task Manager, the dedicated GPU memory went from 1GB before run to about 9.8GB for the most time during run, peaking at 10.2GB. So pretty close to the 11GB limit of my 2080Ti it seems.

Re: Whisper – open source speech recognition by OpenAI

#300

Post by harry8 »

Can you plug this into a computer on your premises to get speech recognition without amazon, apple or google's cloud (or any other cloud) involvement?

Right now I decline all speed recognition because I don't want orwellian listening devices in my house or pocket and haven't seen an answer. (Also haven't been too bothered about speech command interfaces to bother with a load of research - lazy me).

Post reply on HN