Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

261–270 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#261

Post by coder543 »

lunixbochs wrote: post #238
coder543 wrote: post #194

Earlier quoted context omitted.

“CPU” isn’t necessarily the benchmark, though. Most smartphones going back years have ML inference accelerators built in, and both Intel and AMD are starting to build in instructions to accelerate inference. Apple’s M1 and M2 have the same inference accelerator hardware as their phones and tablets. The question is whether this model is a good fit for those inference accelerators, and how well it works there, or how w…

Ok, my test harness is ready. My A40 box will be busy until later tonight, but on an NVIDIA A2 [1], this is the batchsize=1 throughput I'm seeing. Common Voice, default Whisper settings, card is staying at 97-100% utilization: tiny.en: ~18 sec/sec base.en: ~14 sec/sec small.en: ~6 sec sec/sec medium.en: ~2.2 sec/sec large: ~1.0 sec/sec (fairly wide variance when ramping up as this is slow to process individual clips)…

Isn’t the A2 much weaker than a 3090? So those results are promising.

EDIT: for what it's worth, Nvidia rated the A2 at 18 TFLOPS of FP16, and Apple rates the current A16 Neural Engine at 17 TFLOPS of FP16. I'm sure it's not an "apples to apples" comparison.

Re: Whisper – open source speech recognition by OpenAI

#262

Post by capableweb »

Snitch-Thursday wrote: post #91
TaylorAlexander wrote: post #31

Hey this looks great! I like to record audio notes while driving in my car after work, to kind of decompress my thoughts from the day. But I never go back and listen as they can be long and meandering. Sometimes in the audio log I will sum up my thoughts, but this might be 20 minutes in and hard to find. I really wish I had transcriptions so I could easily scan the full contents. I have tried Mozilla Deepspeech (I do…

Google's recorder app for android will let you record audio files and make some transcriptions, right on the device.

Is that application actually doing on-device transcription? Under "Data safety" on the Google Play page it says "This app may share these data types with third parties: Audio" which doesn't exactly instill confidence that my audio will 100% always stay on my device. It also says "Data is encrypted in transit" but if data stays on the device, why it has to be "encrypted in transit"? There should be no transit at all.

Re: Whisper – open source speech recognition by OpenAI

#264

Post by TaylorAlexander »

no1youknowz wrote: post #214

This is awesome. But I really want the other way. To be able to give it text and hear the speech. A TTS (text to speech). As a language learner, the ability to create my own sentences (based on existing ones I have, in changing a word here or there). Would be amazing. How long till we have this I wonder. I know I could use a service to do this currently. But having something running locally, I'd prefer. Hopefully som…

I suspect this is coming. I mean we do have decent text to speech systems already, but in this vein of “we used neural networks and now it’s very very good” you can imagine that with something like GPT-3, to extend it they could use this speech to text system so you could speak to it for input, and then a natural progression is that it can use text to speech to return the output, so you just have a voice oriented conversational system.

So I think TTS is a logical part of the system. I also think that there are peculiarities of voice interaction that aren’t captured in text training datasets, so they would need to do some fine tuning on actual voice conversation to make it feel natural.

All in due time I suppose.

Re: Whisper – open source speech recognition by OpenAI

#265

Post by bambax »

noreally_ wrote: post #229

A notebook is available to try with your microphone on Colab here: https://colab.research.google.com/drive/1nBZ-pDIaIi3N1DIIXvJ... I'm surprised by the quality on non-English languages, given that 80+% of the training data is English, and the rest is split between tens of languages.

Thanks! I played with this in French and posted the results as replies to this comment: https://news.ycombinator.com/item?id=32928643

It's sometimes close to perfect, and sometimes goes off the rail; I think that maybe the model tries to establish some sort of consistency for each sentence; if starts wrong for the first few words of a sentence, it can't build the rest properly.

But it's super fun.

Re: Whisper – open source speech recognition by OpenAI

#267

Post by nshm »

eatsyourtacos wrote: post #165

Can this be used as a real-time transcription or is it too slow for that? Curious what anyone is using these days for a real-time transcription. It doesn't have to be perfect, but just good enough. My kids watch some youtube vidoes where people will make a mod where it converts them talking to text then look for keywords and spawn a boss in Terraria if you say the wrong keyword etc. I made a clone of that with the .N…

Try https://github.com/alphacep/vosk-api/blob/master/csharp/demo...

Re: Whisper – open source speech recognition by OpenAI

#268

Post by WiSaGaN »

ma2rten wrote: post #201
petercooper wrote: post #148

Just tested this on some developer podcasts which usually fail hard given they're full of technical jargon, brand names, etc. Whisper is a revolution! It's picking up terms like Heroku, DigitalOcean, GitHub, ECS, AWS, etc. and capitalizing properly - something nothing else did unless you provided a whole pile of guiding vocabulary.

Did these podcasts have transcripts? You might be inadvertently evaluating it on data that it was trained on, which is basically cheating. Even if not, it might be trained on similar podcasts. Judging how good these kinds of models are is really hard.

True. The test should only be done on the material released after the model.

Re: Whisper – open source speech recognition by OpenAI

#269

Post by [deleted] »

space_fountain wrote: post #109
soheil wrote: post #107

Earlier quoted context omitted.

Quoted post unavailable.

This isn't exactly a hard story to fact check. There is 0 evidence for this in either the reddit thread or really anywhere? If they were willing to lie about the company name why not just lie about the beef in their burgers it would be equally scandalous

[deleted]

Re: Whisper – open source speech recognition by OpenAI

#270

Post by TaylorAlexander »

eatsyourtacos wrote: post #165

Can this be used as a real-time transcription or is it too slow for that? Curious what anyone is using these days for a real-time transcription. It doesn't have to be perfect, but just good enough. My kids watch some youtube vidoes where people will make a mod where it converts them talking to text then look for keywords and spawn a boss in Terraria if you say the wrong keyword etc. I made a clone of that with the .N…

The base model seems to run faster than real time on my machine. The “medium” model is larger and runs more slowly - roughly real time or maybe slightly slower.
Post reply on HN