Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

461–470 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#461

Post by MacsHeadroom »

rvz wrote: post #361
shpx wrote: post #259

We shouldn't call this open source. The model definition + the data is the source code. The model weights are a compilation artifact. > The source code must be the preferred form in which a programmer would modify the program. [...] Intermediate forms such as the output of a preprocessor or translator are not allowed. > https://opensource.org/osd If I asked a programmer from OpenAI to modify the model to better suppo…

Yes. It just like calling the release of compiled closed binary blobs as 'open source' even when the source of reproducing the compiled output is unavailable. > If I asked a programmer from OpenAI to modify the model to better support Japanese speakers from Hokkaido, their "preferred form" of the model's source code would include the 680,000 hours of audio used to train the model. Precisely. These 'users' lifting the…

>You will still be contacting OpenAI for support or to add support for another language and they will be the ones able to modify the model.

This isn't quite correct. The model weights are all you need to fine tune the data on your own with your own audio.

Without the original training set this still isn't open source. But you aren't powerless to modify the model without the original training set.

Re: Whisper – open source speech recognition by OpenAI

#462

Post by simmanian »

hijp wrote: post #88

Anyone get it running on m1 mac? I keep getting `ModuleNotFoundError: No module named 'setuptools.command.build'`

I got it working inside a docker container on my M1 MBP. FWIW, I'm having my $180 tinyminimicro PC run a translation task while my M1 MBP runs a transcription task with the same audio input. So far, the PC is actually outputting results a lot faster than the MBP. Interesting results.

Re: Whisper – open source speech recognition by OpenAI

#464

Post by aidenn0 »

macrolocal wrote: post #457
aidenn0 wrote: post #197

Earlier quoted context omitted.

Ran a few other songs through it and found one obvious mistranscription: "He's the bedroom cosmic rocker" (should be "He's the veteran cosmic rocker" in Veteran Cosmic Rocker by The Moody Blues) I also noticed that it's a little on the conservative side for detecting speech; all songs were missing at least part of one line.

For what it's worth, even the large model balks on Easy (Aesop Rock), eg. "Fountainheads spittle sniglets quicker than quidditch seekers snatch golden snitches." becomes "Stirred up out mids bittles, snicklets, cricket and quidditch seekers net golden snitches." ¯\_(ツ)_/¯

Large was not obviously better than medium when I tried it. My impression was that it tended to fit more to a language model than the sounds heard, which corrected some errors and introduced some others, but I didn't try a lot of songs because large won't run on my GPU.

Re: Whisper – open source speech recognition by OpenAI

#465

Post by bergenty »

synergy20 wrote: post #463

is there a high quality text to speech equivalent project like this?

Seriously, when I first landed on the page without reading anything else I thought it was text to speech with the “micro machine” example and I was floored. The speech to text is obviously mind blowing too.

Re: Whisper – open source speech recognition by OpenAI

#466

Post by code51 »

First off, it seems that the model can easily run on M1/M2 with minor modification. However `aten::_index_put_impl_` operator is current not supported and fallback always slows things down quite a lot.

Second, is there a bug with how the script processes incoming audio segments? For a short 4 second clip, what I got was:

> [00:00.000 --> 00:03.760] Okay, Eunice, travel plans. I need to be in New York on Monday, L.A. on Tuesday, New York on Wednesday, L.A. on Thursday. You're knocking Friday. Got it?

> [00:03.760 --> 00:28.760] Got it.

However the final segment should have been shy of 1 second. It mistakenly thinks the last segment was 25 seconds long and makes you wait for processing.

Re: Whisper – open source speech recognition by OpenAI

#467

Post by owenpalmer »

danso wrote: post #417

This is an astonishing package. Every AI voice-to-text model I've tried on "The Wire's" famous "fuck" scene [0] usually fails, because the youtube clip's audio quality is bad and it's a scene with virtually no dialogue except breathing and "Fuck". But Whisper returned impressive results [1] [0] https://www.youtube.com/watch?v=DS6pE88Xg3s [1] $ yt-dlp --extract-audio --audio-format mp3 -o wire-fuck.mp3 https://www.you…

nsfw

Re: Whisper – open source speech recognition by OpenAI

#468

Post by yencabulator »

gnramires wrote: post #175
anigbrowl wrote: post #116

Earlier quoted context omitted.

No it isn't. That just means 2-3% of your content needs to be double-checked by a person at the audio level, saving huge amounts of time - equally true of human transcription, in which individual words are often [UNINTELLIGEBLE]. Would you want to review this fully before going into court, absolutely - because you'd want to play the recording to a jury for emotional impact. Can you rely on it when you want to quickly…

I think an [UNINTELLIGIBLE] indication would be a great addition to automatic transcription systems.

Google Voice voicemail transcription used to do this, with varying levels of gray. It seems that feature is gone, now.

Re: Whisper – open source speech recognition by OpenAI

#469

Post by yencabulator »

anigbrowl wrote: post #65
mmastrac wrote: post #20

Earlier quoted context omitted.

Large is 3GB to save everyone a click. Tiny is 72MB.

That's unexpectedly lightweight - enough to run in some phones.

However, https://github.com/openai/whisper#available-models-and-langu... says requires ~1 GB VRAM.

Re: Whisper – open source speech recognition by OpenAI

#470

Post by sjsdaiuasgdia »

I was comparing a batch of transcriptions between these models and vosk, and noticed that the medium.en model produces some weird results compared to the others. I've seen a number of loops with one word or a small sequence of words repeating several times. It seems more prone to output that reads like nonsense than the others.

More troubling is a short audio clip that got a few full sentences back, several times the text length that comes back from the other models or vosk. The content of the sentences is extremely far from the audio content. The best alignment I can find is the first word of medium.en's interpretation is somewhat phonetically similar to the audio.

The small.en model doesn't show these behaviors, at least in this data set.

Post reply on HN