FFmpeg 8.0 adds Whisper support • Hacker News Board

Re: FFmpeg 8.0 adds Whisper support

#21

Post by yorwba » Wed, Aug 13, 2025, 10:46 AM UTC

jeroenhd wrote: post #18

boutell wrote: post #4

Shut off the broken bot filter so we can read it please

Check out commit 13ce36fef98a3f4e6d8360c24d6b8434cbb8869b from https://git.ffmpeg.org/ffmpeg.git if your web browser doesn't support Javascript. The linked page is just a git viewer for that specific commit.

Or read the documentation for the new whisper filter: https://ffmpeg.org/ffmpeg-filters.html#whisper-1

Re: FFmpeg 8.0 adds Whisper support

#22

Post by regularfry » Wed, Aug 13, 2025, 10:47 AM UTC

zzsshh wrote: post #3

Does this finally enable dynamically generating subtitles for movies with AI?

If you have enough processing power. Without a GPU it's going to lag.

Re: FFmpeg 8.0 adds Whisper support

#23

Post by londons_explore » Wed, Aug 13, 2025, 10:48 AM UTC

Does this have the ability to edit historic words as more info becomes available?

Eg. If I say "I scream", it sounds phonetically identical to "Ice cream".

Yet the transcription of "I scream is the best dessert" makes a lot less sense than "Ice cream is the best dessert".

Doing this seems necessary to have both low latency and high accuracy, and things like transcription on android do that and you can see the adjusting guesses as you talk.

Re: FFmpeg 8.0 adds Whisper support

#24

Post by jeroenhd » Wed, Aug 13, 2025, 10:49 AM UTC

londons_explore wrote: post #16

diggan wrote: post #9

Earlier quoted context omitted.

Took my iPhone 12 Mini a whole of 0.1 seconds to pass it. What hardware/OS are you using?

Took about 30 secs for me (5 yr old intel cpu). Looked like there was a progress bar, but it didn't progress. Maybe the difficulty varies depending on IP address?

Anubis has config for that: https://anubis.techaro.lol/docs/admin/policies#request-weigh...

It's up to the site admin to configure it that way, but it's possible some IP ranges/user agents are more often used by bots and therefore have an increased weight.

For old browsers there's also an option to use meta refresh instead of JS (https://anubis.techaro.lol/docs/admin/configuration/challeng...) but that's quite a recent addition and not enabled by default.

Re: FFmpeg 8.0 adds Whisper support

#25

Post by QuantumNomad_ » Wed, Aug 13, 2025, 10:50 AM UTC

boutell wrote: post #4

Shut off the broken bot filter so we can read it please

Archived snapshots of the linked page:

https://web.archive.org/web/20250813104007/https://code.ffmp...

https://archive.is/dmj17

You can read it on one of these without having to pass that specific bot check

Re: FFmpeg 8.0 adds Whisper support

#26

Post by jeroenhd » Wed, Aug 13, 2025, 10:51 AM UTC

yorwba wrote: post #21

jeroenhd wrote: post #18

Earlier quoted context omitted.

Check out commit 13ce36fef98a3f4e6d8360c24d6b8434cbb8869b from https://git.ffmpeg.org/ffmpeg.git if your web browser doesn't support Javascript. The linked page is just a git viewer for that specific commit.

Or read the documentation for the new whisper filter: https://ffmpeg.org/ffmpeg-filters.html#whisper-1

That also works, I assumed the ffmpeg website would also be behind Anubis if the git server is, but it doesn't actually seem to be.

Re: FFmpeg 8.0 adds Whisper support

#27

Post by re » Wed, Aug 13, 2025, 10:51 AM UTC

I've been playing with whisper to try to do local transcription of long videos, but one issue I've found is that long (>15 seconds) spans without any speech tend to send it into a hallucination loops that it often can't recover from. I wonder if, with direct integration into ffmpeg, they will be able to configure it in a way that can improve that situation.

Re: FFmpeg 8.0 adds Whisper support

#28

Post by cess11 » Wed, Aug 13, 2025, 10:51 AM UTC

voxadam wrote: post #8

Am I correct in understanding that Whisper is a speech recognition AI model originally created by OpenAI? https://en.wikipedia.org/wiki/Whisper_(speech_recognition_sy...

Kind of, it's a family of audio transcription models.

https://huggingface.co/search/full-text?q=whisper

Re: FFmpeg 8.0 adds Whisper support

#29

Post by oezi » Wed, Aug 13, 2025, 10:51 AM UTC

Maxious wrote: post #17

voxadam wrote: post #8

Am I correct in understanding that Whisper is a speech recognition AI model originally created by OpenAI? https://en.wikipedia.org/wiki/Whisper_(speech_recognition_sy...

yep, there's a c++ implementation to run it https://github.com/ggml-org/whisper.cpp

Isn't WhisperX the canonical choice for running Whisper?

Re: FFmpeg 8.0 adds Whisper support

#30

Post by voxadam » Wed, Aug 13, 2025, 10:53 AM UTC

johnisgood wrote: post #15

voxadam wrote: post #8

Am I correct in understanding that Whisper is a speech recognition AI model originally created by OpenAI? https://en.wikipedia.org/wiki/Whisper_(speech_recognition_sy...

Yes. From the documentation: > It runs automatic speech recognition using the OpenAI's Whisper model.

Thanks, I was being tripped up by DDOS protection on code.ffmpeg.org for a minute and couldn't read the patch. The combo of Firefox and the fact that Quantum/Lumen/CenturyLink seems to get off by rotating my dynamic IP for no reason occasionally triggers various DDOS protections schemes.