Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

41–50 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#41

Post by wongarsu »

The5thElephant wrote: post #26

How is it Apple, Google, or Microsoft are not further ahead of the game on speech recognition like this? They have the resources to hire the best ML researchers and throw tons of computing hours at it, yet Siri, Google, and Cortana continue to struggle to get anywhere near this level of comprehension.

Siri and Cortana have to run at least in real time, with reasonable compute resources. Probably faster than real time when the audio gets shipped off to the cloud and transcribed there. This model can't do that (in the "large" version, which the examples use).

Also, you are comparing Whisper's highlight reel with everyday performance of other models. Nobody shows their weaknesses in their highlight reel.

Re: Whisper – open source speech recognition by OpenAI

#42

Post by ALittleLight »

wongarsu wrote: post #16
samstave wrote: post #7

AI speech recognition FN scares the heck out of me... for so many reasons. But one that really pisses me off is not being able to turn it off on the iphone, and the fact that aside from "hidden cameras in my airBnB" -- soon we will have to worry about secret listening machines EVERYWHERE

"Secret listening machines everywhere" was a pretty big thing in East Germany. It's also the central theme of the movie The Lives of Others. Of course, the ability to scale this more cheaply (throwing more compute at it, instead of more people) is somewhat scary, but it's not really introducing a new capability. Especially since you still have to do something with the transcript. An AirBnB landlord who reads the tran…

I think it's a new capability to add good speech to text, search, and models that can understand and process text. You have microphones recording speech everywhere, models turning that speech into easily searchable text, and something like GPT-3 reading all the speech and raising red flags for any transgressive idea you please.

Re: Whisper – open source speech recognition by OpenAI

#43

Post by newhaus1994 »

andy_xor_andrew wrote: post #27

Hold on, it does not only speech recognition, but also language translation, in the same model? What an interesting approach. What benefits does this have over having two dedicated models, one for speech-to-text, and another for translation? It just seems so odd, given the problems of speech-to-text and Spanish-to-English seems so different from one another (in terms of the problem domain). Seems so unusual to have b…

My understanding is that multi-modal models are the primary focus of OpenAI right now, due to their stated goal of achieving AGI. This product is probably better thought of as an offshoot of their work to create a fully generalizable model, rather than a specific attempt to provide translation/transcription services.

Re: Whisper – open source speech recognition by OpenAI

#44

Post by alex_marchant »

wongarsu wrote: post #41
The5thElephant wrote: post #26

How is it Apple, Google, or Microsoft are not further ahead of the game on speech recognition like this? They have the resources to hire the best ML researchers and throw tons of computing hours at it, yet Siri, Google, and Cortana continue to struggle to get anywhere near this level of comprehension.

Siri and Cortana have to run at least in real time, with reasonable compute resources. Probably faster than real time when the audio gets shipped off to the cloud and transcribed there. This model can't do that (in the "large" version, which the examples use). Also, you are comparing Whisper's highlight reel with everyday performance of other models. Nobody shows their weaknesses in their highlight reel.

Siri until ios 15 was done in the cloud IIRC.

Re: Whisper – open source speech recognition by OpenAI

#45

Post by nik_s »

solarmist wrote: post #36
nik_s wrote: post #34

I just tested the model [1] using an RTX3090, trying to translate a french text I found here [2]. Some observations: - The full translation of the 6:22 minute video takes about 22 seconds (17x real time) - It recognizes the language by default (and did a good job to recognize it was french audio) - MIT License [3]! - The quality of the transcription is good, but not perfect. - The quality of the translation (if you d…

Is it translation or transcription? Or both? Both, wow. This is really interesting.

It can do both - I've edited my original post to show the translation task.

Re: Whisper – open source speech recognition by OpenAI

#46

Post by samstave »

ALittleLight wrote: post #42
wongarsu wrote: post #16

Earlier quoted context omitted.

"Secret listening machines everywhere" was a pretty big thing in East Germany. It's also the central theme of the movie The Lives of Others. Of course, the ability to scale this more cheaply (throwing more compute at it, instead of more people) is somewhat scary, but it's not really introducing a new capability. Especially since you still have to do something with the transcript. An AirBnB landlord who reads the tran…

I think it's a new capability to add good speech to text, search, and models that can understand and process text. You have microphones recording speech everywhere, models turning that speech into easily searchable text, and something like GPT-3 reading all the speech and raising red flags for any transgressive idea you please.

Yes, and if you want AI that is searching for “dissenters” we shall soon have “speech police” or tickets or some format of authoritarian punitive actions powered by this

Re: Whisper – open source speech recognition by OpenAI

#47

Post by _plg_ »

pen2l wrote: post #32
bambax wrote: post #29

Earlier quoted context omitted.

The French version is a little contrived. The speaker is a native speaker, but the text is obviously the result of a translation from English to French, not idiomatic French. I will try to put the code to the test, see how it goes.

Interesting, I'm a non-native French speaker, the original French piece struck me as being entirely normal (but maybe it was just the perfect French accent that swayed me). Can you please point out what he said which wasn't idiomatic or naturally-worded French?

At the start, the "Nous établissons" part, for example. You wouldn't write that if you were starting scratch from French.

Re: Whisper – open source speech recognition by OpenAI

#48

Post by not_math »

pen2l wrote: post #32
bambax wrote: post #29

Earlier quoted context omitted.

The French version is a little contrived. The speaker is a native speaker, but the text is obviously the result of a translation from English to French, not idiomatic French. I will try to put the code to the test, see how it goes.

Interesting, I'm a non-native French speaker, the original French piece struck me as being entirely normal (but maybe it was just the perfect French accent that swayed me). Can you please point out what he said which wasn't idiomatic or naturally-worded French?

You can see from the transcript where the model made some errors, for example:

> We distribute as a free software the source code for our models and for the inference [...]

Should be

> We are open-sourcing models and inference code [...]

Another example

> We establish that the use of such a number of data is such a diversity and the reason why our system is able [...]

Should be

> We show that the use of such a large and diverse dataset leads to improved robustness [...]

Re: Whisper – open source speech recognition by OpenAI

#49

Post by beastman82 »

The5thElephant wrote: post #26

How is it Apple, Google, or Microsoft are not further ahead of the game on speech recognition like this? They have the resources to hire the best ML researchers and throw tons of computing hours at it, yet Siri, Google, and Cortana continue to struggle to get anywhere near this level of comprehension.

In my unmeasured empirical observation Google has amazing speech recognition

Re: Whisper – open source speech recognition by OpenAI

#50

Post by jffry »

wongarsu wrote: post #16
samstave wrote: post #7

AI speech recognition FN scares the heck out of me... for so many reasons. But one that really pisses me off is not being able to turn it off on the iphone, and the fact that aside from "hidden cameras in my airBnB" -- soon we will have to worry about secret listening machines EVERYWHERE

"Secret listening machines everywhere" was a pretty big thing in East Germany. It's also the central theme of the movie The Lives of Others. Of course, the ability to scale this more cheaply (throwing more compute at it, instead of more people) is somewhat scary, but it's not really introducing a new capability. Especially since you still have to do something with the transcript. An AirBnB landlord who reads the tran…

I'd argue that cheap, pervasive, always-on surveillance with a backlog of searchable transcriptions is a qualitatively different capability.
Post reply on HN