Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

191–200 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#191

Post by kyriakos »

hadlock wrote: post #106
thfuran wrote: post #85

Earlier quoted context omitted.

>~97% accuracy over hour-long conversations. I'm sure it's been an absolute godsend for law enforcement 97% accuracy means roughly three or four errors per minute of speech. That seems potentially extremely problematic for something like law enforcement use where decisions with significant impact on people's day and/or life might be made on the basis of "evidence".

Microsoft announced their voice transcription technology a couple years ago and were also touting ~97-98% accuracy which was actually better than human transcription error rates. The errors are usually in part people garbling their own speech, or they move their head while talking and the microphone misses a syllable. Anything in that error bar would probably fall under "reasonable doubt"

If its anything like Microsoft teams transcription I doubt the 97%+ accuracy.

Re: Whisper – open source speech recognition by OpenAI

#192

Post by agnos »

lynguist wrote: post #132

How can I use this (or something similar) for live translation? I don't mind if there's a 30s delay. As in I don't want to input a file, I want to input the microphone sound.

Would also like to know this. It looks like they're processing the audio file in 30 second chunks, so a naive approach of keeping a buffer of 30-second input stream chunks and just continually writing to an output .mp3 could work...

Re: Whisper – open source speech recognition by OpenAI

#193

Post by mewse-hn »

mewse-hn wrote: post #188

I know this isn't a tech support forum but maybe someone here knows. I'm attempting the sample python code from the github and almost get a transcription running on my work laptop without a GPU, but I run into this error message: >>> result = whisper.decode(model, mel, options) Traceback (most recent call last): [snip] RuntimeError: "slow_conv2d_cpu" not implemented for 'Half' It looks like a Torch error, is there so…

I seem to have worked around it by tweaking the "options" line from the sample code to this:

>>> options = whisper.DecodingOptions(fp16=False)

Re: Whisper – open source speech recognition by OpenAI

#194

Post by coder543 »

lunixbochs wrote: post #190
coder543 wrote: post #138

Earlier quoted context omitted.

Someone else in this thread[0] said Whisper was running at 17x real time for them. So, even a weak machine might be able to do an acceptable approximation of real time with Whisper. Also, I feel like shipping to the cloud and back has been shown to be just as fast as on device transcription in a lot of scenarios. Doing it on device is primarily a benefit for privacy and offline, not necessarily latency. (Although, in…

17x realtime on a 3090 I did some basic tests on CPU, the "small" Whisper model is in the ballpark of 0.5x realtime, which is probably not great for interactive use. My models in Talon run closer to 100x realtime on CPU.

“CPU” isn’t necessarily the benchmark, though. Most smartphones going back years have ML inference accelerators built in, and both Intel and AMD are starting to build in instructions to accelerate inference. Apple’s M1 and M2 have the same inference accelerator hardware as their phones and tablets. The question is whether this model is a good fit for those inference accelerators, and how well it works there, or how well it works running on the integrated GPUs these devices all have.

Brute forcing the model with just traditional CPU instructions is fine, but… obviously going to be pretty slow.

I have no experience on the accuracy of Talon, but I’ve heard that most open source models are basically overfit to the test datasets… so their posted accuracy is often misleading. If Whisper is substantially better in the real world, that’s the important thing, but I have no idea if that’s the case.

Re: Whisper – open source speech recognition by OpenAI

#195

Post by formerly_proven »

thfuran wrote: post #101
golem14 wrote: post #99

Earlier quoted context omitted.

One would think that the few crucial bits of information gleaned are listened to manually, and the machine translation is not the only thing the judge or a jury sees.

You have absolutely ruined someone's day way before they're sitting in front of a jury.

Stuff like that is a very good tell that someone has zero experience with law enforcement.

Re: Whisper – open source speech recognition by OpenAI

#196

Post by Smaug123 »

dceddia wrote: post #187
Smaug123 wrote: post #164

Earlier quoted context omitted.

Yep, same for me, on M1 after enabling MPS (with `model.to("mps")`) it just either SIGSEGV or SIGABRTs every time with that line. The extremely unclean nature of the abort is making it hard to debug :(

I noticed the size seems to correspond to the model. With a large model, the error is tensor . With tiny, it's tensor , and with medium it's tensor . It also seems like a bad thing that those are f16's but the "expected" data is f32.

I'm giving up for the night, but https://github.com/Smaug123/whisper/pull/1/files at least contains the setup instructions that may help others get to this point. Got it working on the GPU, but it's… much much slower than the CPU? Presumably due to the 'aten::repeat_interleave.self_int' CPU fallback.

Also hitting a nice little PyTorch bug:

> File "/Users/patrick/Documents/GitHub/whisper/whisper/decoding.py", line 388, in apply logits[:, self.tokenizer.encode(" ") + [self.tokenizer.eot]] = -np.inf

> RuntimeError: dst_.nbytes() >= dst_byte_offset INTERNAL ASSERT FAILED at "/Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/mps/operations/Copy.mm":200, please report a bug to PyTorch.

Re: Whisper – open source speech recognition by OpenAI

#197

Post by aidenn0 »

aidenn0 wrote: post #122

I just threw a random rock MP3 at it, and a first readthrough shows no transcription errors; this is quite good. Now I just want OCR that's even 50% as good as this...

Ran a few other songs through it and found one obvious mistranscription:

"He's the bedroom cosmic rocker" (should be "He's the veteran cosmic rocker" in Veteran Cosmic Rocker by The Moody Blues)

I also noticed that it's a little on the conservative side for detecting speech; all songs were missing at least part of one line.

Re: Whisper – open source speech recognition by OpenAI

#199

Post by mewse-hn »

kif wrote: post #115
kif wrote: post #113

Earlier quoted context omitted.

I got requirements installed, but then when running the Python example, I get: RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'

Probably need to pass some kind of options when initializing. The command itself works fine, just shows a warning: warnings.warn("FP16 is not supported on CPU; using FP32 instead")

using this in the sample code worked for me:

>>> options = whisper.DecodingOptions(fp16=False)

Re: Whisper – open source speech recognition by OpenAI

#200

Post by pessimizer »

soheil wrote: post #178
pessimizer wrote: post #170

Earlier quoted context omitted.

Something being possible to do isn't enough evidence for rational people to believe that it happened. From my perspective, it's possible that you're Iron Mike Tyson, or that you died after your last comment and this one was posted by the assassin who killed you.

What? I never said it's evidence that it did happen, please don't make things up. I just pointed out the evidence provided to refute the claim is possibly invalid.

You haven't offered any evidence is the point.
Post reply on HN