Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

Post reply on HN
Print view View original

501–508 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#501

Post by james-revisoai »

liminalsunset wrote: post #69

I really wish I had this about half a year ago when I was building a tool to automatically turn online school lectures into searchable, clickable transcripts (kind of like YouTube or EdX transcripts). I was originally using Adobe Premiere Pro's speech to text to do it, and wrote Python to convert its output to the Hyperaudio format on GitHub. With this, I can totally skip all of that step and this is fully open sourc…

You still interested in this? I'd be keen to chat to you, worked on a searchable transcript provider for educational youtube videos (likewise, unfortunately pre-whisper, so I did a lot of work with sentence completion perplexity and rpunct to try and improve transcript quality from youtube automatic transcriptions). Can be contacted at revision.ai and demo what we were able to do till now, would be great to hear your thoughts.

Re: Whisper – open source speech recognition by OpenAI

#502

Post by lunixbochs »

ma2rten wrote: post #185
lunixbochs wrote: post #94

Earlier quoted context omitted.

I suspect Whisper is more robust than other "SOTA" models, but this release is likely leaving a fair bit of accuracy on the table considering the amount of resources OpenAI is capable of throwing at training it. Comparing the readily available test sets from the paper to some of my personal robust models (for the Talon models, this is greedy decoding, no language model): Talon Talon Talon Whisper wav2vec 2.0 28M 300M…

I'm looking forward to your comparison. It's really hard to make sense of how good this model actually is without being an expert in the area.

Just posted results here: https://twitter.com/lunixbochs/status/1574848899897884672

Re: Whisper – open source speech recognition by OpenAI

#503

Post by arbol »

ricopags wrote: post #206
suyash wrote: post #169

Earlier quoted context omitted.

This is only one side of the coin, you still need really good models for Speech Synthesis and then be able to have it all working in almost real time, ideally locally on device.

As far as TTS goes, Mycroft.ai[0] has released a decent offline one. [0] https://mycroft.ai/

I'm pretty sure mycroft sends your speech snippets to Google for processing so it's not exactly offline.

https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz...

I'm currently trying to setup a deepspeech server on my raspberry pi to see if it works ok for commanding spotify.

Edit: just realised you said `TTS` not `STT`

Re: Whisper – open source speech recognition by OpenAI

#504

Post by selfmodruntime »

JohnFen wrote: post #166
selfmodruntime wrote: post #141

Earlier quoted context omitted.

I've worked with similar technology in the law enforcement space and the software is never used to make decisions. You can make out critical timestamps in conversations and a law enforcement officer will always manually confirm the software's assessments.

Given that law enforcement has made similar claims about technology use in the past that turned out to be false, I have no faith in this claim.

In all honesty, this is the correct mindset to have. I have limited expertise in this topic, and you should be aware that other law enforcement agencies probably do not handle this the same way.

Re: Whisper – open source speech recognition by OpenAI

#505

Post by BurntNorton »

NaturalPhallacy wrote: post #340
ramblerman wrote: post #339

Earlier quoted context omitted.

--device "cuda"

My version of pytorch didn't have CUDA. I had to install conda to get it, and now it's currently installing. Whatever the default version that `pip install git+ https://github.com/openai/whisper.git ` grabbed didn't include it by default.

I installed Whisper (and, I thought all the needed dependencies), and had it running on my M1 Max MacBook Pro with 64 GB ram, but it ran TERRIBLY slowly... taking an hour to do a couple of minutes...

I found this thread and wondered if Whisper was accessing all the cores or the gpu, so I've spent a couple of hours trying to get whisper to access the gpu - following the points made in this thread, and googling how to install via brew the various components.

Long story short, I keep getting an error message

"RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU."

or when I set --device to gpu, it get the error: "RuntimeError: don't know how to restore data location of torch.storage._UntypedStorage (tagged with gpu)"

it's been a looong time since I wrote any code (remember basic?), so realise I may be missing a lot here!!

does anyone have any pointers?

thanks!

edit: I'm now trying it one more time after trying to set the cpu using this line:

map_location=torch.device('gpu')

and I get this message as whisper begins: ~/opt/anaconda3/lib/python3.9/site-packages/whisper/transcribe.py:78: UserWarning: FP16 is not supported on CPU; using FP32 instead warnings.warn("FP16 is not supported on CPU; using FP32 instead")

then I wait for whisper to do it's magic ...tho it looks like it will remain very slow...

Re: Whisper – open source speech recognition by OpenAI

#506

Post by marcelfahle »

danso wrote: post #417

This is an astonishing package. Every AI voice-to-text model I've tried on "The Wire's" famous "fuck" scene [0] usually fails, because the youtube clip's audio quality is bad and it's a scene with virtually no dialogue except breathing and "Fuck". But Whisper returned impressive results [1] [0] https://www.youtube.com/watch?v=DS6pE88Xg3s [1] $ yt-dlp --extract-audio --audio-format mp3 -o wire-fuck.mp3 https://www.you…

As interesting as it is funny. Great benchmark! Here's the rev.ai output for comparison:

  Speaker 0    00:00:12    Oh, fuck motherfucker. Okay. Fuck, fuck, fuck, fuck, fuck, fuck, fuck, fuck. 
 My little fuck.  
  Speaker 1    00:02:10    Oh, fuck. Oh, fuck,  
  Speaker 0    00:02:25    Fuck, fuck, fuck, fuck, fuck, fuck, fuck, fuck my motherfucker.  
  Speaker 1    00:02:53    Fucking a.  
  Speaker 0    00:02:54    Mm-hmm.  motherfucker. Fuck me. Um,

Re: Whisper – open source speech recognition by OpenAI

#507

Post by samat »

harry8 wrote: post #300

Can you plug this into a computer on your premises to get speech recognition without amazon, apple or google's cloud (or any other cloud) involvement? Right now I decline all speed recognition because I don't want orwellian listening devices in my house or pocket and haven't seen an answer. (Also haven't been too bothered about speech command interfaces to bother with a load of research - lazy me).

Btw, Apple's speech recognition can work completely offline, on-device. Not sure about Google or Microsoft, though.

Re: Whisper – open source speech recognition by OpenAI

#508

Post by Vecr »

code51 wrote: post #466

First off, it seems that the model can easily run on M1/M2 with minor modification. However `aten::_index_put_impl_` operator is current not supported and fallback always slows things down quite a lot. Second, is there a bug with how the script processes incoming audio segments? For a short 4 second clip, what I got was: > [00:00.000 --> 00:03.760] Okay, Eunice, travel plans. I need to be in New York on Monday, L.A.…

The system only works on 30 second chunks, the system needs padding (and the CLI does the padding for you).
Post reply on HN