Okay that should be my last reboot for a while, I've got it ready how I'd like it for now, feel free to give any feedback!
Show HN: I made a free transcription service powered by Whisper AI
121–130 of 135 posts
Re: Show HN: I made a free transcription service powered by Whisper AI
#122Earlier quoted context omitted.
Hey, glad I could be of use. The problem with Whisper is that it needs a lot of GPU. Actually my Mac can't even use my GPU so right away I had to get it up on a server, but Whisper is so powerful and it's so amazing that it's open source I am surprised nobody did this yet. I could see them charging for it but may as well use it anyways, the other services are insanely expensive ($10/h?!) and I don't really like their…
Nothing has come back from the two that I tried (one medium in French, the other large in Spanish), meaning no change on the page since I uploaded them an hour an a half ago. I loaded the page again in another tab, though, and after a few seconds "finishedProcessing" appeared under the form. I suspect that means something. On Firefox 102.4.0esr, also uBlock Origin.
It's probably due to me rebooting to load new code, I will have a way to send a signal to the frontend to inform them but not implemented atm
Re: Show HN: I made a free transcription service powered by Whisper AI
#123Earlier quoted context omitted.
Nothing has come back from the two that I tried (one medium in French, the other large in Spanish), meaning no change on the page since I uploaded them an hour an a half ago. I loaded the page again in another tab, though, and after a few seconds "finishedProcessing" appeared under the form. I suspect that means something. On Firefox 102.4.0esr, also uBlock Origin.
It's probably due to me rebooting to load new code, I will have a way to send a signal to the frontend to inform them but not implemented atm
I tried it again this morning. I'm getting all of the output properly this time, but it has hung partway through every time I tried.
Re: Show HN: I made a free transcription service powered by Whisper AI
#124Earlier quoted context omitted.
How long would whisper.cpp take to transcribe 2 hours of audio on M1?
Not sure about M1, but on the Macbook Pro 14" with an M1 Max using 8 threads I transcribed a 44 minute podcast in 16 minutes. So about 3x "real time" speed.
What model are you using? I guess large, as my M1 Max takes about 1.4 min for a 4 min file (35% of total time)?
Re: Show HN: I made a free transcription service powered by Whisper AI
#125If somebody wants to run Whisper for dirt cheap, vast.ai is the way to go.
I can usually get an (unverified) 1x RTX 3090 instance for about $0.10/hr, and that processes audio at something like 1.5X speeds. Unverified instances do crash once in a while, but as long as you back up the output every few hours, it's fine, you just set up a new one in case something happens. I wouldn't use this for confidential company meetings, but it's good enough for podcasts, Youtube Videos and other public or semi-public stuff.
Re: Show HN: I made a free transcription service powered by Whisper AI
#126Earlier quoted context omitted.
It's probably due to me rebooting to load new code, I will have a way to send a signal to the frontend to inform them but not implemented atm
I tried it again this morning. I'm getting all of the output properly this time, but it has hung partway through every time I tried.
It's me again. Ran it again, ran perfectly. Thanks for all of your work.
edit: don't know if you'll see this any time soon, but I've had it fail/hang again. You might want to take a hash of uploads, so if the lost connections still end up getting transcribed, if they're reuploaded they won't get transcribed again.
Also I haven't had success in Firefox, only Chromium.
Re: Show HN: I made a free transcription service powered by Whisper AI
#127I’m sorry to self promote again - but: https://whispermemos.com I’m in love with the idea of pressing button on my Lock Screen and getting a perfect transcription in my inbox. Also, just added emoji summarization in email subject, a small visual reminder of what your memo was about. I hope this is useful to someone!
is the app open source? I'm on android, so :c
If you don't mind non-OSS, the Google Recorder App is incredible.
Re: Show HN: I made a free transcription service powered by Whisper AI
#128Earlier quoted context omitted.
What was the performance, resource usage, etc of doing this with large? What's the speed like? I'm still getting spun up on this but base delivers a pretty impressive 5-20x realtime on my RTX 3090. I haven't gotten around to trying the larger models and with only 24GB of VRAM I'm not sure what kind of success I'll have anyway... In my case the goal was to actually generate tweets based on XYZ. As I've already said th…
You can run the larger models just fine on a 3090. Large takes about 10G for transcribing English. For a 1:17 file it takes: 6s for base.en, I think 2s to load the model based on the sound of my power supply. 33s for large, I think 11s of which is loading the model. Varies a lot with how dense the audio file is, this was me giving a talk so not the fastest and quite clean audio. While I saw near perfect or perfect pe…
> based on the sound of my power supply
Hah, I love that - "benchmark by fan speed".
Good to know - I've tried large and it works but in my case I'm using whisper-asr-webservice[0] which loads the configured model for each of the workers on startup. I have some prior experience with Gunicorn and other WSGI implementations so there's some playing around and benchmarking to be done on the configured number of workers as the GPU utilization of Whisper is a little spiky and whisper-asr-webservice does file format conversion on CPU via ffmpeg. Default was two workers, is now one but I've found as many as four with base can really improve overall utilization, response time, and scale (which certainly won't be possible with large).
OPs node+express implementation shells out to Whisper which gives more control (like runtime specification of model) but almost certainly has to end up slower and less efficient in the long run as the model is obviously loaded from scratch on each invocation. I'm front-ending whisper-asr-webservice with traefik so I could certainly do something like having two separate instances (one for base, another for large) at different URL paths but like I said I need to do some playing around with it. The other issue is if this is being made available to the public I doubt I'd be comfortable without front-ending the entire thing with Cloudflare (or similar) and Cloudflare (and others) have things like 100s timeouts for final HTTP response (Websockets could get around this).
Thanks for providing the Slim Shady examples, as a life-long hip hop enthusiast I'm not offended by the content in the slightest.
Re: Show HN: I made a free transcription service powered by Whisper AI
#129Earlier quoted context omitted.
Not sure about M1, but on the Macbook Pro 14" with an M1 Max using 8 threads I transcribed a 44 minute podcast in 16 minutes. So about 3x "real time" speed.
What model are you using? I guess large, as my M1 Max takes about 1.4 min for a 4 min file (35% of total time)?
Yep, large model.
Re: Show HN: I made a free transcription service powered by Whisper AI
#130Earlier quoted context omitted.
> I'm just wary about restarting the server because I haven't setup a way to be able to reboot without losing the websockets Wait what. Not being able to safely restart the server sounds like a disaster waiting to happen.
> losing the websockets users would lose the session and have to start over, not the end of the world
I'm not even using sessions just localstorage and websockets lol.