Congratulations on taking this to completion and announcing here! Love your approach to this!
Show HN: I made a free transcription service powered by Whisper AI
51–60 of 135 posts
Re: Show HN: I made a free transcription service powered by Whisper AI
#52I recently tried Whisper to transcribe our local Seattle Fire Department radio scanner -- unfortunately it was not reliable enough for my use case, e.g. "adult male hit by car" gets transcribed as "don't mail it by car". I imagine future models will allow the user to input some context to disambiguate. Like if I could give it the audio along with the context "Seattle Fire Department and EMS radio traffic", it would b…
The issue here is for most radio systems you end up with about 3 kHz of effective audio bandwidth (sampling). Most ASR/STT models are trained on at least 16 kHz audio. Did you try a telephony oriented model like aspire or similar? They’re trained on sort-of 8 kHz audio and might work better. I tried something similar for my SDR feeds and gave up because it’s just too challenging and niche - the sampling, the jargon,…
> the sampling, the jargon, the 10 codes, the background noise, static on analog systems/drop outs on digital systems, rate of speech, etc
Is the reduced bandwidth really the most significant problem? Naively I'd think everything else you mentioned would matter a lot more, I'm curious how much you experimented with that specifically.
Re: Show HN: I made a free transcription service powered by Whisper AI
#53I recently tried Whisper to transcribe our local Seattle Fire Department radio scanner -- unfortunately it was not reliable enough for my use case, e.g. "adult male hit by car" gets transcribed as "don't mail it by car". I imagine future models will allow the user to input some context to disambiguate. Like if I could give it the audio along with the context "Seattle Fire Department and EMS radio traffic", it would b…
Was there a big difference in accuracy depending on which model you used?
Yes, large was by far the best, but still not accurate enough that I'd be willing to put it into a fully automated pipeline. It would have gotten it right probably 75% of the time. Anything other than the large model was far too bad to even think about using.
Re: Show HN: I made a free transcription service powered by Whisper AI
#54Earlier quoted context omitted.
Was there a big difference in accuracy depending on which model you used?
Yes, large was by far the best, but still not accurate enough that I'd be willing to put it into a fully automated pipeline. It would have gotten it right probably 75% of the time. Anything other than the large model was far too bad to even think about using.
Whisper does pretty well, even with background music and things like that, I think you're working with a pretty weird subsection of recorded audio that won't work, for that edge case to work you'll very likely need to train your own model.
Re: Show HN: I made a free transcription service powered by Whisper AI
#55Earlier quoted context omitted.
The issue here is for most radio systems you end up with about 3 kHz of effective audio bandwidth (sampling). Most ASR/STT models are trained on at least 16 kHz audio. Did you try a telephony oriented model like aspire or similar? They’re trained on sort-of 8 kHz audio and might work better. I tried something similar for my SDR feeds and gave up because it’s just too challenging and niche - the sampling, the jargon,…
> the sampling, the jargon, the 10 codes, the background noise, static on analog systems/drop outs on digital systems, rate of speech, etc Is the reduced bandwidth really the most significant problem? Naively I'd think everything else you mentioned would matter a lot more, I'm curious how much you experimented with that specifically.
When it all comes together it's kind of a nightmare for an ASR model. There were plenty of times in reviewing the recordings and ASR output where I'd listen to the audio and have no idea what they said.
I'm not sure which contributes most but I know from my prior experiences with ASR for telephony even clean speech on pristine connections does much worse with models trained on 16 kHz being fed native 8 kHz audio that gets resampled.
I've done some early work with Whisper in the telephony domain (transcribing voicemails on Asterisk and Freeswitch) and the accuracy already seems to be quite a bit worse.
Re: Show HN: I made a free transcription service powered by Whisper AI
#56Free startup idea: Use Whisper with pyannote-audio[0]’s speaker diarization. Upload a recording, get back a multi-speaker annotated transcription. Make a JSON API and I’ll be your first customer. [0] https://github.com/pyannote/pyannote-audio
And you expect the API to be free? If not why not use one of a million other such services?
Re: Show HN: I made a free transcription service powered by Whisper AI
#57There’s a lot of startups starting in the space offering transcription. Read.ai - https://www.read.ai/transcription Provides transcription & diarization and the bot integrates into your calendar. It joins all your meetings for zoom, teams, meet, webex, tracks talk time, gives recommendations, etc. It’s amazing how quickly this space is moving. Particularly, with the increase in remote work. Soon you’ll be able to sea…
Yeah I was paying $100/month for transcription services and turns out Whisper with the large model was much more accurate, and I didn't like the UI, I much prefer to just use this app as opposed to the paid service, and I chose it because it was the cheapest by far ($100/30h) as opposed to most of the other paid services which were $10 an hour which to me was a bit much really. But Whisper is really a game changer I don't know how those companies stay in business really.
Re: Show HN: I made a free transcription service powered by Whisper AI
#58Free startup idea: Use Whisper with pyannote-audio[0]’s speaker diarization. Upload a recording, get back a multi-speaker annotated transcription. Make a JSON API and I’ll be your first customer. [0] https://github.com/pyannote/pyannote-audio
It's not as if people aren't trying to do that: https://github.com/openai/whisper/discussions/264 I tried out this notebook about a month ago, and it was rough . After spending an evening improving it, I got everything "working", but pyannote was not reliable. I tried it against an hour-ish audio sample, and I found no way to tune pyannote to keep track of ~10 speakers over the course of that audio. It would identify…
Speaker recognition is another piece that isn't usually as high a priority as recognizing the speech.
Re: Show HN: I made a free transcription service powered by Whisper AI
#59Earlier quoted context omitted.
Was there a big difference in accuracy depending on which model you used?
Yes, large was by far the best, but still not accurate enough that I'd be willing to put it into a fully automated pipeline. It would have gotten it right probably 75% of the time. Anything other than the large model was far too bad to even think about using.
What was the performance, resource usage, etc of doing this with large? What's the speed like?
I'm still getting spun up on this but base delivers a pretty impressive 5-20x realtime on my RTX 3090. I haven't gotten around to trying the larger models and with only 24GB of VRAM I'm not sure what kind of success I'll have anyway...
In my case the goal was to actually generate tweets based on XYZ. As I've already said there were serious technical challenges so I abandoned the project but I was also a little concerned about the privacy, safety, etc issues of realtime or near-realtime reporting on public safety activity. I also streamed to broadcastify and it really seems like they insert artificial delay because of these concerns.
Re: Show HN: I made a free transcription service powered by Whisper AI
#60Congratulations on taking this to completion and announcing here! Love your approach to this!
Thanks! I wrote it for myself over a weekend and have really enjoyed it ever since I'm glad others were able to get something out of it! It seems to run pretty well but I have some improvements planned, first is I will take the whisper output and feed it to you when you're in the queue so you can see them progressing. Will be pretty trivial to implement but I am feeling bored in the queue at the site atm so that is the next killer feature lol