Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

341–350 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#341

Post by darepublic »

pen2l wrote: post #5

Neat, https://github.com/openai/whisper - they have open-sourced it, even the model weights, so they are living up to their name in this instance. The 4 examples are stunningly good (the examples have speakers with heavy accents, speaking in foreign language, speaking with dynamic background noise, etc.), this is far and away better than anything else I've seen. Will be super curious to see other folks trying it out…

> Neat, https://github.com/openai/whisper - they have open-sourced it, even the model weights, so they are living up to their name in this instance.

Perhaps it will encourage people to add voice command to their apps, which can be sent to gpt3

Re: Whisper – open source speech recognition by OpenAI

#342

Post by pabs3 »

pen2l wrote: post #5

Neat, https://github.com/openai/whisper - they have open-sourced it, even the model weights, so they are living up to their name in this instance. The 4 examples are stunningly good (the examples have speakers with heavy accents, speaking in foreign language, speaking with dynamic background noise, etc.), this is far and away better than anything else I've seen. Will be super curious to see other folks trying it out…

Is the training dataset and code open too?

Re: Whisper – open source speech recognition by OpenAI

#343

Post by coder543 »

fxtentacle wrote: post #237
coder543 wrote: post #158

Earlier quoted context omitted.

You just said the models pretty much all work the same way, then you said doing what I described won't help. I'm confused. Apple and Google both offer real time, on device transcription these days, so something clearly works. And if you say the models already all do this, then running it 30x as often isn't a problem anyways, since again... people are used to that. I doubt people run online transcription for long peri…

I meant that most research that has been released in papers or code recently uses the same architecture. But all of those research papers use something different than Apple and Google. As for running the AI 30x, on current hardware that'll make it slower than realtime. Plus all of those 1GB+ models won't fit into a phone anyway.

> Plus all of those 1GB+ models won't fit into a phone anyway.

I don't think that's a requirement here. I've been playing with Whisper tonight, and even the tiny model drastically outperformed Siri dictation for me in my testing. YMMV, of course.

Re: Whisper – open source speech recognition by OpenAI

#344

Post by frognumber »

anigbrowl wrote: post #213
wging wrote: post #127

Earlier quoted context omitted.

Doesn't it mean 100% of your content needs to be double-checked? You can't easily identify which 2-3% of your content has errors. I'm aware that errors are more likely when the model is less confident of its predictions, but that shouldn't be enough. (edit for clarification: errors are not always something like "[UNINTELLIGIBLE]", where the system knows it doesn't know; they can also be misrecognitions that the syste…

By the time you're prosecuting someone in court, yes of course you double, triple, quadruple check everything. That's why lawyers get paid the big bucks (for now...). But yes you can identify which content probably has errors and flag it as such. Look, I have decades of experience dealing with human speech, and not just as an editor - I can trace the human voice from neural impulses in Broca's region through the phys…

What tools do you use to do this? I once hacked together an editor like this maybe a decade ago -- edit speech as text from OCR -- and sorely need one now.

Alignment of video to text is a big problem for me too.

Re: Whisper – open source speech recognition by OpenAI

#345

Post by magicalhippo »

magicalhippo wrote: post #278
mwlp wrote: post #233

Super impressive. I tested it on a Japanese streamer whose enunciation isn't exactly perfect and it did a decent job: https://www.youtube.com/watch?v=ROiOU1scaNA [00:00.000 --> 00:06.500] Since the last one started, the number of times I've eaten has decreased. [00:06.500 --> 00:11.000] If I get too carried away with the last one, I'll get hungry and do it. [00:11.000 --> 00:14.500] I don't have time to eat. [00:15.5…

It's struggling with Norwegian. Which I guess isn't shocking. The large model performs a fair bit better than the small, though neither is "good". Though I assume the amount of Norwegian it has been exposed to is fairly limited, so in that light I'm actually impressed as well. I tried it on a news segment from the radio[1], this is the large model output: [00:14.000 --> 00:17.200] En skamløs krenking av FN pakten. [0…

Re-reading the transcription, I guess I was a bit harsh by saying it's not "good". It gets most of it right, but it keeps messing up some key words. Like "regnstyr" (not a word) rather than "reinsdyr" (reindeer), or "Dagsnytten" rather than "Dagsnytt 18".

It also didn't handle the hanging "... menn", instead thinking it was the start of the following sentence. Almost everyone would understand it was the end of the sentence based on the context.

The double-A vs Å is not an issue as it's the same letter, double-A is the older form.

The small model was considerably worse than the large one though.

Re: Whisper – open source speech recognition by OpenAI

#346

Post by pabs3 »

manishsharan wrote: post #76

Oh this is a relief to have something opensource in this field. I had using Mozilla Deepspeech for transcribing my voice notes , often with hilarious to incomprehensible results. DeepSpeech is dead ; so I will be sure to check this out.

DeepSpeech got spun out of Mozilla to coqui.ai and they are continuing the open nature of the project.

Re: Whisper – open source speech recognition by OpenAI

#347

Post by aGHz »

bambax wrote: post #61
pen2l wrote: post #32

Earlier quoted context omitted.

Interesting, I'm a non-native French speaker, the original French piece struck me as being entirely normal (but maybe it was just the perfect French accent that swayed me). Can you please point out what he said which wasn't idiomatic or naturally-worded French?

Little details. The second sentence is really bizarre: > Nous établissons que l'utilisation de données d'un tel nombre et d'une telle diversité est la raison pour laquelle le système est à même de comprendre de nombreux accents... It doesn't sound natural at all. An idiomatic formulation would be more along the lines of: Le recours à un corpus [de données] si riche et varié est ce qui permet au système de comprendre…

That's interesting, as a québécois I don't agree with any of this. The only thing that raised an eyebrow was "est à même de", but if turns out it's just another way of saying "capable de", I guess it's simply not a common idiom around here. Aside from that, I found the wording flowed well even if I personally would've phrased it differently.

Re: Whisper – open source speech recognition by OpenAI

#348

Post by NaturalPhallacy »

dreamer7 wrote: post #338
NaturalPhallacy wrote: post #319

Earlier quoted context omitted.

It's a micromachines ad from the '80s. He talked like that in all of them! As for speed, to a computer we don't talk very fast, not even that guy. I wonder if it could handle Rap God by Eminem....Let's find out!

Did you find out :D?

It was doing it slowly, but hadn't got to the insane bit when I killed it to try and get it working with CUDA, so I had to do some digging and it turns out I need a version of pytorch with CUDA enabled, and so I had to go and install Anaconda, and now now conda is stuck trying to "solve" my environment to install pytorch with CUDA.

So...probably?

Pre-post edit: I can't get it to work.

I've installed pytorch with cuda via pip3, installed the nVidia toolkit and it doesn't see it:

>>> import torch >>> torch.cuda.is_available() False

I've wasted like an hour and a half on it now. I'm not a python dev, and don't have any ML experience so this was just for fun and now it's not anymore.

Re: Whisper – open source speech recognition by OpenAI

#349

Post by pabs3 »

shpx wrote: post #259

We shouldn't call this open source. The model definition + the data is the source code. The model weights are a compilation artifact. > The source code must be the preferred form in which a programmer would modify the program. [...] Intermediate forms such as the output of a preprocessor or translator are not allowed. > https://opensource.org/osd If I asked a programmer from OpenAI to modify the model to better suppo…

The Debian deep learning team's machine learning policy would call this a "toxic candy" model:

https://salsa.debian.org/deeplearning-team/ml-policy

BTW, wouldn't you take the existing model and do additional Hokkaido Japanese speaker training on top of it, rather than retraining the model from scratch?

Re: Whisper – open source speech recognition by OpenAI

#350

Post by magicalhippo »

dceddia wrote: post #161
Smaug123 wrote: post #105

Earlier quoted context omitted.

I'm still not successfully using the GPU, but it's working decently quickly (with the base model - it's incredibly slow to use the Large model) using just the CPU. I'm going to have to check what magic stable-diffusion is doing to enable the GPU :(

There's a --device flag you can pass. I've been trying to get `--device cuda` to work on my Windows machine and it's saying that torch wasn't compiled with CUDA. Trying to figure out what's going on there. And on the M1, supposedly PyTorch has support for hardware acceleration using MPS (Metal Performance Shaders, announced here https://pytorch.org/blog/introducing-accelerated-pytorch-tra... ) but when I tried `--dev…

> I've been trying to get `--device cuda` to work on my Windows machine and it's saying that torch wasn't compiled with CUDA.

I struggled with the same. Here's what worked for me:

Use pip to uninstall pytorch first, should be "pip uninstall torch" or similar.

Find the CUDA version you got installed[1]. Go to PyTorch get started page[2] and use their guide/wizard to generate the pip string, and run that. I had to change pip3 to pip FWIW, and with Cuda 11.6 installed I ended up with "pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116".

After that I could use --device cuda, and the difference was immense. On my 2080Ti it went from roughly an hour for a minute with large model, to 10-20 seconds.

[1]: https://stackoverflow.com/a/55717476

[2]: https://pytorch.org/get-started/locally/

Post reply on HN