Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

Post reply on HN
Print view View original

181–190 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#181

Post by smallpipe »

0cf8612b2e1e wrote: post #174
Fluorescence wrote: post #110

Earlier quoted context omitted.

It makes me curious about how human subtitlers or even scriptwriters choose to transcribe intentionally ambiguous speech, puns and narratively important mishearings. It's like you need to subtitle what is heard not what is said. Do those born profoundly deaf specifically study word sounds in order to understand/create puns, rhymes and such so they don't need assistance understanding narrative mishearings? It must fee…

The quality of subtitles implies that almost no effort is being put into their creation. Watch even a high budget movie/TV show and be aghast at how frequently they diverge.

A good subtitle isn't a perfect copy of what was said.

Re: FFmpeg 8.0 adds Whisper support

#182

Post by threecheese »

jokethrowaway wrote: post #159
kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

whisper is definitely nice, but it's a bit too slow. Having subtitles and transcription for everything is great - but Nemo Parakeet (pretty much whisper by nvidia) completely changed how I interact with the computer. It enables dictation that actually works and it's as fast as you can think. I also have a set of scripts which just wait for voice commands and do things. I can pipe the results to an LLM, run commands,…

Would you share the scripts?

Re: FFmpeg 8.0 adds Whisper support

#184

Post by codedokode »

kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

Kdeenlive also supports auto-generating subtitles which need some editing, but it is faster than create them from scratch. Actually I would be happy even with a simple voice detector so that I don't have to set the timings manually.

Re: FFmpeg 8.0 adds Whisper support

#187

Post by dotancohen »

porridgeraisin wrote: post #113

I had a small bash pipeline for doing this until now. ffmpeg -f pulse -i "$(pactl get-default-source)" -t 5 -f wav -ar 16000 -ac 1 -c:a pcm_s16le - \ | ./main - \ | head -2 \ | tail -1 \ | cut -d] -f2 \ | awk '{$1=$1};1' The reading from mic part (-f pulse, pactl...) is linux-specific rest of it should be cross platform. The `main` executable is the whisper.cpp executable (see whisper.cpp github readme, it's just the…

  > which I then....
Yes, please, go on...

Re: FFmpeg 8.0 adds Whisper support

#188

Post by codedokode »

abound wrote: post #106
yvdriess wrote: post #94

Earlier quoted context omitted.

A good opportunity to point people to the paper with my favorite title of all time: "How to wreck a nice beach you sing calm incense" https://dl.acm.org/doi/10.1145/1040830.1040898

For folks like me puzzling over what the correct transcription of the title should be, I think it's "How to recognize speech using common sense"

I only got the "How to recognize" part. Also I think "using" should sound more like "you zinc" than "you sing".

Re: FFmpeg 8.0 adds Whisper support

#189

Post by neckro23 »

trenchpilgrim wrote: post #83
donatj wrote: post #69

I know nothing about Whisper, is this usable for automated translation? I own a couple very old and as far as I'm aware never translated Japanese movies. I don't speak Japanese but I'd love to watch them. A couple years ago I had been negotiating with a guy on Fiver to translate them. At his usual rate-per-minute of footage it would have cost thousands of dollars but I'd negotiated him down to a couple hundred before…

Whisper has quite bad issues with hallucination. It will inject sentences that were never said in the audio. It's decent for classification but poor at transcription.

Pre-processing with a vocal extraction model (bs-rofomer or similar) helps a lot with the hallucinations, especially with poor quality sources.

Re: FFmpeg 8.0 adds Whisper support

#190

Post by xyse53 »

yvdriess wrote: post #94
londons_explore wrote: post #23

Does this have the ability to edit historic words as more info becomes available? Eg. If I say "I scream", it sounds phonetically identical to "Ice cream". Yet the transcription of "I scream is the best dessert" makes a lot less sense than "Ice cream is the best dessert". Doing this seems necessary to have both low latency and high accuracy, and things like transcription on android do that and you can see the adjusti…

A good opportunity to point people to the paper with my favorite title of all time: "How to wreck a nice beach you sing calm incense" https://dl.acm.org/doi/10.1145/1040830.1040898

My favorite is:

"Threesomes, with and without blame"

https://dl.acm.org/doi/10.1145/1570506.1570511

(From a professor I worked with a bit in grad school)

Post reply on HN