Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

61–70 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#61
post #32
post #29

Earlier quoted context omitted.

The French version is a little contrived. The speaker is a native speaker, but the text is obviously the result of a translation from English to French, not idiomatic French. I will try to put the code to the test, see how it goes.

Interesting, I'm a non-native French speaker, the original French piece struck me as being entirely normal (but maybe it was just the perfect French accent that swayed me). Can you please point out what he said which wasn't idiomatic or naturally-worded French?

Little details. The second sentence is really bizarre:

> Nous établissons que l'utilisation de données d'un tel nombre et d'une telle diversité est la raison pour laquelle le système est à même de comprendre de nombreux accents...

It doesn't sound natural at all. An idiomatic formulation would be more along the lines of:

Le recours à un corpus [de données] si riche et varié est ce qui permet au système de comprendre de nombreux accents (With 'corpus', 'données' is implied.)

Of course this is just an example, and I'm sure other French speakers could come up with a different wording, but "données d'un tel nombre et d'une telle diversité" sounds really wrong.

This is also weird and convoluted:

> Nous distribuons en tant que logiciel libre le code source pour nos modèles et pour l'inférence, afin que ceux-ci puissent servir comme un point de départ pour construire des applications utiles

It should at least be "le code source DE nos modèles" and "servir DE point de départ", and "en tant que logiciel libre" should placed at the end of the proposition (after 'inférence').

Also, "construire" isn't used for code but for buildings, and "applications utiles" is unusual, because "utiles" (useful) is assumed. "...pour le développement de nouvelles applications" would sound more French.

Re: Whisper – open source speech recognition by OpenAI

#64
post #29
post #5

Neat, https://github.com/openai/whisper - they have open-sourced it, even the model weights, so they are living up to their name in this instance. The 4 examples are stunningly good (the examples have speakers with heavy accents, speaking in foreign language, speaking with dynamic background noise, etc.), this is far and away better than anything else I've seen. Will be super curious to see other folks trying it out…

The French version is a little contrived. The speaker is a native speaker, but the text is obviously the result of a translation from English to French, not idiomatic French. I will try to put the code to the test, see how it goes.

I'm interested in building something with this to aid my own French learning. Would love to read your findings if you end up posting it somewhere like twitter/blog!

Re: Whisper – open source speech recognition by OpenAI

#65

Earlier quoted context omitted.

(Model weights from https://github.com/openai/whisper/blob/main/whisper/__init__... ) "tiny.en": " https://openaipublic.azureedge.net/main/whisper/models/d3dd5... " "tiny": " https://openaipublic.azureedge.net/main/whisper/models/65147... " "base.en": " https://openaipublic.azureedge.net/main/whisper/models/25a85... " "base": " https://openaipublic.azureedge.net/main/whisper/models/ed3a0... " "small.en": " https://op…

Large is 3GB to save everyone a click. Tiny is 72MB.

That's unexpectedly lightweight - enough to run in some phones.

Re: Whisper – open source speech recognition by OpenAI

#66
This is so cool! I was just speaking to a non-technical family member about privacy concerns around using "OK Google" and the like. They responded inquiring about "private" alternatives, to which my answer was "I'm not aware of good ones that give you that level of accuracy and convenience."

Perhaps this development along with continued optimization and device compute power increases will lead us into a near-future where things like Mycroft devices and cellphones could have local-only speech-to-text and translation capabilities which are accurate even with environmental background noise variations encountered IRL.

Great work OpenAI team!

Re: Whisper – open source speech recognition by OpenAI

#67

The model output can be tweaked to produce audio embeddings (akin to BERT for text embeddings and CLIP for image embeddings), which can lead to some interesting applications as the previous two examples have demonstrated.

What do you mean exactly by audio embeddings?

Re: Whisper – open source speech recognition by OpenAI

#68
post #35

Be wary of using this model - the licensing of this model seems sketchy. Several of the datasets used for training like WSJ and TED-LIUM have clear non-commercial clauses. I'm not a lawyer but releasing a model as "MIT" seems dubious, and hopefully OpenAI has paid for the appropriate licenses during training as they are no longer a research-only non profit.

This is a big dispute right now: OpenAI and other AI companies generally take the position that models learning from data does not make the output of the models a derivative work of that data. For example, GitHub Co-pilot uses all publicly available GitHub code regardless of license, and DALLE-2/StableDiffusion/etc use lots of non-free images. I don't think this has been challenged in court yet, and I'm very curious to see what happens when it is.

Re: Whisper – open source speech recognition by OpenAI

#69
I really wish I had this about half a year ago when I was building a tool to automatically turn online school lectures into searchable, clickable transcripts (kind of like YouTube or EdX transcripts).

I was originally using Adobe Premiere Pro's speech to text to do it, and wrote Python to convert its output to the Hyperaudio format on GitHub. With this, I can totally skip all of that step and this is fully open source, too.

App idea:

Build an app that takes a video and uses Hyperaudio or a similar project to add a clickable and searchable transcript (clicking in transcript seeks video)

Post reply on HN