Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

151–160 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#151

Post by Tenoke »

Snitch-Thursday wrote: post #91
TaylorAlexander wrote: post #31

Hey this looks great! I like to record audio notes while driving in my car after work, to kind of decompress my thoughts from the day. But I never go back and listen as they can be long and meandering. Sometimes in the audio log I will sum up my thoughts, but this might be 20 minutes in and hard to find. I really wish I had transcriptions so I could easily scan the full contents. I have tried Mozilla Deepspeech (I do…

Google's recorder app for android will let you record audio files and make some transcriptions, right on the device.

I just tested it and it was pretty mediocre at least with my accent. I can definitely benefit from a decent app for quick note recording with a button press->transcribe->upload to gdrive/good UI app for later grepping.

Re: Whisper – open source speech recognition by OpenAI

#152

Post by jefftk »

soheil wrote: post #118
space_fountain wrote: post #109

Earlier quoted context omitted.

This isn't exactly a hard story to fact check. There is 0 evidence for this in either the reddit thread or really anywhere? If they were willing to lie about the company name why not just lie about the beef in their burgers it would be equally scandalous

The company name could be 100% legit, there is nothing stopping you from a forming a company with that name and not even sell beef.

If this was more than an urban legend someone would be able to dig up a company with this name and some indication that McD was working with them.

Re: Whisper – open source speech recognition by OpenAI

#153

Post by nshm »

lunixbochs wrote: post #94
gok wrote: post #19

Comparing this model's word error rates to the state of the art [1] on a few common test sets: Whisper SoTA LibriSpeech test-clean 2.7% 1.8% LibriSpeech test-other 5.6% 2.9% Switchboard 13.1% 4.9% CallHome 15.8% 9.5% The authors do explicitly state that they're trying to do a lot of fancy new stuff here, like be multilingual, rather than pursuing just accuracy. [1] https://github.com/syhw/wer_are_we

I suspect Whisper is more robust than other "SOTA" models, but this release is likely leaving a fair bit of accuracy on the table considering the amount of resources OpenAI is capable of throwing at training it. Comparing the readily available test sets from the paper to some of my personal robust models (for the Talon models, this is greedy decoding, no language model): Talon Talon Talon Whisper wav2vec 2.0 28M 300M…

It is interesting how they compare with wav2vec2 instead of nemo conformer (which is more accurate) in Table 2.

Re: Whisper – open source speech recognition by OpenAI

#154

Post by smusamashah »

How well does it do for technical and domain oriented speech? For example I have audio recordings of a senior explaining some very technical aspects of our software. Will it understand the technical terms in that speech?

I guess I will need to download and run on it to see how correct it is.

Re: Whisper – open source speech recognition by OpenAI

#155

Post by fxtentacle »

coder543 wrote: post #149
fxtentacle wrote: post #114

Earlier quoted context omitted.

This AI has a 30 second delay on the audio processing because it needs to be able to "look into the future" to get these good results. That 30s delay would be unacceptable for Siri/Google/Cortana.

A lot of models we currently use seem to do the same thing. The model will transcribe a "best effort" interpretation in real time, then as you can continue speaking, you'll see it go back and make corrections. I'm sure you can feed the first X seconds you have into the model, followed by (30-X) seconds of silence, and it will do real time transcription just fine... it would be weird if this broke anything. Then, as y…

Yes, that's because Whisper - like pretty much all of them - uses a Transformer encoder with Attention layers. And the Attention layers learn to look into the future.

And yes, what you describe could be done. But no, it won't reduce latency that much, because the model itself learns to delay the prediction w.r.t. the audio stream. That's why ASR-generated subtitles usually need to be re-aligned after the speech recognition step. And that's why there is research such as the FastEmit paper to prevent that, but then it is a trade-off between latency and quality again.

Also, running your "low-latency" model with 1s chunks means you now need to evaluate the AI 30x as often as if you'd be using 30s chunks.

Re: Whisper – open source speech recognition by OpenAI

#156

Post by datalopers »

anigbrowl wrote: post #116
thfuran wrote: post #85

Earlier quoted context omitted.

>~97% accuracy over hour-long conversations. I'm sure it's been an absolute godsend for law enforcement 97% accuracy means roughly three or four errors per minute of speech. That seems potentially extremely problematic for something like law enforcement use where decisions with significant impact on people's day and/or life might be made on the basis of "evidence".

No it isn't. That just means 2-3% of your content needs to be double-checked by a person at the audio level, saving huge amounts of time - equally true of human transcription, in which individual words are often [UNINTELLIGEBLE]. Would you want to review this fully before going into court, absolutely - because you'd want to play the recording to a jury for emotional impact. Can you rely on it when you want to quickly…

If you know which 2-3% are the false positives, you have a very lucrative business model.

Re: Whisper – open source speech recognition by OpenAI

#157

Post by xkapastel »

jfoster wrote: post #98
bigyikes wrote: post #82

Earlier quoted context omitted.

I don’t see how GPT-3 is any more dangerous than Stable Diffusion, Photoshop, that fake news website the crazy person you’re friends with on Facebook really likes, or any of the number of other tools and services that can be used to generate or spread fake information.

All of your examples are limited in some way, but GPT-3 wouldn't have any meaningful limits. Stable Diffusion: Marks images as AI-generated. (invisible watermark, but still, it's there) Photoshop: Requires time & effort from a human. Fake news website: Requires time & effort from a human.

I wouldn't really say Stable Diffusion marks images as AI-generated. There's a script in the Stable Diffusion repository that will do that, but it's not connected to the model itself in a meaningful way. I use Stable Diffusion a lot and I've never touched this script.

https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...

Re: Whisper – open source speech recognition by OpenAI

#158

Post by coder543 »

fxtentacle wrote: post #155
coder543 wrote: post #149

Earlier quoted context omitted.

A lot of models we currently use seem to do the same thing. The model will transcribe a "best effort" interpretation in real time, then as you can continue speaking, you'll see it go back and make corrections. I'm sure you can feed the first X seconds you have into the model, followed by (30-X) seconds of silence, and it will do real time transcription just fine... it would be weird if this broke anything. Then, as y…

Yes, that's because Whisper - like pretty much all of them - uses a Transformer encoder with Attention layers. And the Attention layers learn to look into the future. And yes, what you describe could be done. But no, it won't reduce latency that much, because the model itself learns to delay the prediction w.r.t. the audio stream. That's why ASR-generated subtitles usually need to be re-aligned after the speech recog…

You just said the models pretty much all work the same way, then you said doing what I described won't help. I'm confused. Apple and Google both offer real time, on device transcription these days, so something clearly works. And if you say the models already all do this, then running it 30x as often isn't a problem anyways, since again... people are used to that.

I doubt people run online transcription for long periods of time on their phone very often, so the battery impact is irrelevant, and the model is ideally running (mostly) on a low power, high performance inference accelerator anyways, which is common to many SoCs these days.

Re: Whisper – open source speech recognition by OpenAI

#159

Post by petercooper »

jefftk wrote: post #68
emcq wrote: post #35

Be wary of using this model - the licensing of this model seems sketchy. Several of the datasets used for training like WSJ and TED-LIUM have clear non-commercial clauses. I'm not a lawyer but releasing a model as "MIT" seems dubious, and hopefully OpenAI has paid for the appropriate licenses during training as they are no longer a research-only non profit.

This is a big dispute right now: OpenAI and other AI companies generally take the position that models learning from data does not make the output of the models a derivative work of that data. For example, GitHub Co-pilot uses all publicly available GitHub code regardless of license, and DALLE-2/StableDiffusion/etc use lots of non-free images. I don't think this has been challenged in court yet, and I'm very curious…

I think it might be even less problematic with something like Whisper than with DALLE/SD? Merely consuming data to train a system or create an index is not usually contrary to the law (otherwise Google wouldn't exist) – it's the publication of copyright content that's thorny (and is something you can begin to achieve with results from visual models that include Getty Photos logo, etc.)

I think it'd be a lot harder to make a case for an accurate audio to text transcription being seen to violate the copyright of any of the training material in the way a visual could.

Re: Whisper – open source speech recognition by OpenAI

#160

Post by jeffbee »

beastman82 wrote: post #49
The5thElephant wrote: post #26

How is it Apple, Google, or Microsoft are not further ahead of the game on speech recognition like this? They have the resources to hire the best ML researchers and throw tons of computing hours at it, yet Siri, Google, and Cortana continue to struggle to get anywhere near this level of comprehension.

In my unmeasured empirical observation Google has amazing speech recognition

I tried feeding the four examples from this announcement into Google as dictation inputs and it just sits there blankly. On the JFK speech test file in the repo, Google understands perfectly. The samples in the announcement are clearly outside the capabilities of anything Google has launched publicly, but I don't know how that translates to overall utility in every day applications.
Post reply on HN