Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…
FFmpeg 8.0 adds Whisper support
261–270 of 341 posts
Re: FFmpeg 8.0 adds Whisper support
#262Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…
You don't happen to know a whisper solution that combines diarization with live audio transcription, do you?
whisperx input.mp3 --language en --diarize --output_format vtt --model large-v2
Works a treat for Zoom interviews. Diarization is sometimes a bit off, but generally its correct.Re: FFmpeg 8.0 adds Whisper support
#263 brew install uv
uv tool install openai-whisper
then add ~/.local/bin/ to $PATHRe: FFmpeg 8.0 adds Whisper support
#264https://developer.apple.com/documentation/speech/speechtrans...
https://developer.apple.com/documentation/speech/speechanaly...
https://www.macstories.net/stories/hands-on-how-apples-new-s...
Re: FFmpeg 8.0 adds Whisper support
#265Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…
Can you give an example why it made your life that much better?
10 years ago you'd be searching through random databases to see if someone had synchronized subtitles for the exact copy of the video that you had. Or older lecture videos that don't have transcripts. Many courses had to, in order to comply with federal funding, but not all. And lots of international courses don't have this requirement at all (for example some great introductory CS/maths courses from German + Swiss institutions). Also think about taking this auto generated output and then generating summaries for lecture notes, reading recommendations - this sort of stuff is what LLMs are great at.
You can do some clever things like take the foreign sub, have Whisper also transcribe it and then ask a big model like Gemini to go line by line and check the translation to English. This can include accounting for common transcription errors or idiomatic difference between langauges. I do it in Cursor to keep track of what the model has changed and for easy rollback. It's often good enough to correct mis-heard words that would be garbled through a cheaper model. And you can even query the model to ask about why a particular translation was made and what would be a more natural way to say the same thing. Sometimes it even figures out jokes. It's not a fast or fully automatic process, but the quality can be extremely good if you put some time into reviewing.
Having 90% of this be possible offline/open access is also very impressive. I've not tried newer OSS models like Qwen3 but I imagine it'd do a decent job of the cleanup.
Re: FFmpeg 8.0 adds Whisper support
#266Earlier quoted context omitted.
Writing in the vernacular, I believe it's called. I do something like that if I'm texting. The book "Feersum Endjinn" by Iain M. Banks uses something like this for one of its characters to quite good effect.
Except it forces me to slow down to "decypher" the text and makes the reading labored. I understand the point as it is part of the character, but it is easier to understand someone speaking in that vernacular vs reading the forced misspellings. I definitely don't want to get to the point of being good at reading it though. I wonder if this is how second grade teachers feel reading the class' schoolwork?
I try to limit my use of it to just enough for my accent and way of talking to bleed through. I don't go for full-on phonetics, but I'm often "droppin' my g's and usin' lotsa regional sayin's." It probably helps that the people I text have the same accent I do, though.
Re: FFmpeg 8.0 adds Whisper support
#267Earlier quoted context omitted.
You don't happen to know a whisper solution that combines diarization with live audio transcription, do you?
WhipserX's diarization is great imo: whisperx input.mp3 --language en --diarize --output_format vtt --model large-v2 Works a treat for Zoom interviews. Diarization is sometimes a bit off, but generally its correct.
Thanks but I'm looking for live diarization.
Re: FFmpeg 8.0 adds Whisper support
#268Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…
Re: FFmpeg 8.0 adds Whisper support
#269Earlier quoted context omitted.
> When I'm reading a transcript That's the thing though, subtitles aren't intended as full transcripts . They are intended to allow a wide variety of people to follow the content. A lot of people read slower than they would hear speech. So subtitles often need to condense or rephrase speech to keep pace with the video. The goal is usually to convey meaning clearly within the time available on screen. Not to capture e…
I regularly enable YouTube subtitles. Almost always, they are a 100% verbatim transcription, excluding errors from auto-transcription. I am not annoyed in the slightest, and in fact I very much prefer that they are verbatim. If you are too slow at reading subtitles, you can either slow down the video or train yourself to read faster. Or you can just disable the subtitles.
That's just plain tone deaf, plain and simple. I was not talking about myself, or just youtube. You are not everyone else, your use case is not everyone else their use case. It really isn't that difficult.
Re: FFmpeg 8.0 adds Whisper support
#270Earlier quoted context omitted.
so if "I scream" is in one chunk, and "is the best dessert" is in the next, then there is no way to edit the first chunk to correct the mistake? That seems... suboptimal! I don't think other streaming transcription services have this issue since, whilst they do chunk up the input, past chunks can still be edited. They tend to use "best of N" decoding, so there are always N possible outputs, each with a probability as…
Which other streaming transcription services are you referring to?
The "alternatives" and "confidence" field is the result of the N-best decodings described elsewhere in the thread.