Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

451–460 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#451

Post by perlgeek »

magicalhippo wrote: post #278
mwlp wrote: post #233

Super impressive. I tested it on a Japanese streamer whose enunciation isn't exactly perfect and it did a decent job: https://www.youtube.com/watch?v=ROiOU1scaNA [00:00.000 --> 00:06.500] Since the last one started, the number of times I've eaten has decreased. [00:06.500 --> 00:11.000] If I get too carried away with the last one, I'll get hungry and do it. [00:11.000 --> 00:14.500] I don't have time to eat. [00:15.5…

It's struggling with Norwegian. Which I guess isn't shocking. The large model performs a fair bit better than the small, though neither is "good". Though I assume the amount of Norwegian it has been exposed to is fairly limited, so in that light I'm actually impressed as well. I tried it on a news segment from the radio[1], this is the large model output: [00:14.000 --> 00:17.200] En skamløs krenking av FN pakten. [0…

Everything (and everyone, including myself :D ) seem to struggle with Norwegian, it seems the corpus size is simply too small. And/or maybe the market.

Deepl didn't do any Norwegian last I looked, even though it does most other Germanic languages (including Danish and Swedish).

Duolingo doesn't have a Norwegian class for Germans either, though they do have one with English as the source language.

Re: Whisper – open source speech recognition by OpenAI

#452

Post by vivegi »

wging wrote: post #127
anigbrowl wrote: post #116

Earlier quoted context omitted.

No it isn't. That just means 2-3% of your content needs to be double-checked by a person at the audio level, saving huge amounts of time - equally true of human transcription, in which individual words are often [UNINTELLIGEBLE]. Would you want to review this fully before going into court, absolutely - because you'd want to play the recording to a jury for emotional impact. Can you rely on it when you want to quickly…

Doesn't it mean 100% of your content needs to be double-checked? You can't easily identify which 2-3% of your content has errors. I'm aware that errors are more likely when the model is less confident of its predictions, but that shouldn't be enough. (edit for clarification: errors are not always something like "[UNINTELLIGIBLE]", where the system knows it doesn't know; they can also be misrecognitions that the syste…

You can also use multiple transcription engines and then use mismatches among the text streams to narrow down the % of content that needs to be reviewed. This is quite similar to multi-voting OCR for document images.

The principle is that the engines have different failure modes (hopefully) and therefore the 2-3% error rate of each engine is in different areas of the audio. The key underlying assumption is that the events are mutually exclusive.

With 3 engines, you can use something like 2-of-3 stream matches to override the stream that mismatches.

Re: Whisper – open source speech recognition by OpenAI

#453

Post by 6gvONxR4sf7o »

wging wrote: post #127
anigbrowl wrote: post #116

Earlier quoted context omitted.

No it isn't. That just means 2-3% of your content needs to be double-checked by a person at the audio level, saving huge amounts of time - equally true of human transcription, in which individual words are often [UNINTELLIGEBLE]. Would you want to review this fully before going into court, absolutely - because you'd want to play the recording to a jury for emotional impact. Can you rely on it when you want to quickly…

Doesn't it mean 100% of your content needs to be double-checked? You can't easily identify which 2-3% of your content has errors. I'm aware that errors are more likely when the model is less confident of its predictions, but that shouldn't be enough. (edit for clarification: errors are not always something like "[UNINTELLIGIBLE]", where the system knows it doesn't know; they can also be misrecognitions that the syste…

> I'm aware that errors are more likely when the model is less confident of its predictions, but that shouldn't be enough.

Suppose 90% of the errors are in the 10% where the model is least confident. Then you can review just 10% of your content and take a 2% error rate down to 0.2% error rate.

Re: Whisper – open source speech recognition by OpenAI

#454

Post by archon1410 »

Jnr wrote: post #220

Cool! I am one of the top contributors to the tiny Mozilla Common Voice data-set for my language. The data-set is very small compared to other popular languages and none of the other mentioned data-sets contribute to that language to train the model of Whisper. And even with so little data to train on it still works surprisingly well.

Where do they mention what datasets they've used? I've tried looking at the paper but can't find it.

Re: Whisper – open source speech recognition by OpenAI

#455

Post by alach11 »

mwlp wrote: post #233

Super impressive. I tested it on a Japanese streamer whose enunciation isn't exactly perfect and it did a decent job: https://www.youtube.com/watch?v=ROiOU1scaNA [00:00.000 --> 00:06.500] Since the last one started, the number of times I've eaten has decreased. [00:06.500 --> 00:11.000] If I get too carried away with the last one, I'll get hungry and do it. [00:11.000 --> 00:14.500] I don't have time to eat. [00:15.5…

How long until this gets implemented in Twitch? Real-time subtitles for any stream in the language of your choice?! That would be huge.

Re: Whisper – open source speech recognition by OpenAI

#456

Post by OrangeMusic »

mazork wrote: post #436
bambax wrote: post #61

Earlier quoted context omitted.

Little details. The second sentence is really bizarre: > Nous établissons que l'utilisation de données d'un tel nombre et d'une telle diversité est la raison pour laquelle le système est à même de comprendre de nombreux accents... It doesn't sound natural at all. An idiomatic formulation would be more along the lines of: Le recours à un corpus [de données] si riche et varié est ce qui permet au système de comprendre…

Gonna have to agree with the other reply, as a french-canadian, except for "servir comme un point de départ" which should be "servir de point de départ", that all sounds perfectly fine.

How funny to see that to French people, Quebec french sounds like machine translated english :)

Re: Whisper – open source speech recognition by OpenAI

#457

Post by macrolocal »

aidenn0 wrote: post #197
aidenn0 wrote: post #122

I just threw a random rock MP3 at it, and a first readthrough shows no transcription errors; this is quite good. Now I just want OCR that's even 50% as good as this...

Ran a few other songs through it and found one obvious mistranscription: "He's the bedroom cosmic rocker" (should be "He's the veteran cosmic rocker" in Veteran Cosmic Rocker by The Moody Blues) I also noticed that it's a little on the conservative side for detecting speech; all songs were missing at least part of one line.

For what it's worth, even the large model balks on Easy (Aesop Rock), eg.

"Fountainheads spittle sniglets quicker than quidditch seekers snatch golden snitches."

becomes

"Stirred up out mids bittles, snicklets, cricket and quidditch seekers net golden snitches."

¯\_(ツ)_/¯

Re: Whisper – open source speech recognition by OpenAI

#458

Post by otikik »

_plg_ wrote: post #47
pen2l wrote: post #32

Earlier quoted context omitted.

Interesting, I'm a non-native French speaker, the original French piece struck me as being entirely normal (but maybe it was just the perfect French accent that swayed me). Can you please point out what he said which wasn't idiomatic or naturally-worded French?

At the start, the "Nous établissons" part, for example. You wouldn't write that if you were starting scratch from French.

That's the first thing that I discovered when I visited Paris for the first time.

No one says "Nous", there, ever. Perhaps the politicians, while giving a speech. Everyone else uses the more informal "On".

I felt duped by my French classes.

Re: Whisper – open source speech recognition by OpenAI

#459

Post by wazoox »

bambax wrote: post #253
octref wrote: post #64

Earlier quoted context omitted.

I'm interested in building something with this to aid my own French learning. Would love to read your findings if you end up posting it somewhere like twitter/blog!

Tried again with Blaise Pascal -- the famous fragment of a letter where he says he's sorry he didn't have enough time to make it shorter. Original: > Mes révérends pères, mes lettres n’avaient pas accoutumé de se suivre de si près, ni d’être si étendues. Le peu de temps que j’ai eu a été cause de l’un et de l’autre. Je n’ai fait celle-ci plus longue que parce que je n’ai pas eu le loisir de la faire plus courte. La r…

Depends on the way you're pronouncing it maybe. To be intelligible IMO it must be read differently from a modern text, with well sounding liaisons, and all vowels very distinct: "un" sounds differently from "in", "â" clearly differs from "a", "ai" and "è" from "é" and for instance the "e" in "étendues" must be pronounced, though not loudly.

My test gives that, much better than yours:

Mes *rêverants* pères, mes lettres n'avaient pas accoutumé de se suivre de si près ni d'être si étendues. Le peu de temps que j'ai eu a été cause de l'un et de l'autre. Je n'ai fait celle aussi plus longue que parce que je n'ai pas eu le loisir de *l'af*faire plus courte. La raison qui m'a obligé de me *ra*ter vous est mieux connue qu'à moi. Vos réponses vous réussiss*ez* mal. Vous avez bien fait de changer de méthode. Mais je ne sais si vous avez bien choisi et si le monde ne dira pas que vous avez eu peur des bénédict*eurs*.

Re: Whisper – open source speech recognition by OpenAI

#460

Post by MacsHeadroom »

IceWreck wrote: post #21
StevenWaterman wrote: post #17

Earlier quoted context omitted.

Their models range from 70mb to 3gb. The largest model is smaller than the optimised stable diffusion. Not sure what the inference speed is like, haven't tried it myself yet.

I just tested it myself. Its fast enough on colab, couple of seconds but not sure if its fast enough to transcribe realtime audio yet.

Colab is using one of the larger models. Tiny probably runs in realtime on a single core of an RPi.
Post reply on HN