Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

11–20 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#11
post #3

Does this finally enable dynamically generating subtitles for movies with AI?

Docs say:

    If set, the transcription output will be sent to the specified file or URL
    (use one of the FFmpeg AVIO protocols); otherwise, the output will be logged as info messages.
    The output will also be set in the "lavfi.whisper.text" frame metadata.
    If the destination is a file and it already exists, it will be overwritten.

    @item format
    The destination format string; it could be "text" (only the transcribed text will be sent to the destination), "srt" (subtitle format) or "json".
    Default value: @code{"text"}
I don't know if this can embed the subtitles, but it does support generating accompanying srt files.

Of course, you could already do that by just manually calling whisper on files, but now you don't need to export parts or transformed media files to feed into whisper.

Re: FFmpeg 8.0 adds Whisper support

#15
post #8

Am I correct in understanding that Whisper is a speech recognition AI model originally created by OpenAI? https://en.wikipedia.org/wiki/Whisper_(speech_recognition_sy...

Yes.

From the documentation:

> It runs automatic speech recognition using the OpenAI's Whisper model.

Re: FFmpeg 8.0 adds Whisper support

#16
post #9
post #4

Shut off the broken bot filter so we can read it please

Took my iPhone 12 Mini a whole of 0.1 seconds to pass it. What hardware/OS are you using?

Took about 30 secs for me (5 yr old intel cpu). Looked like there was a progress bar, but it didn't progress. Maybe the difficulty varies depending on IP address?

Re: FFmpeg 8.0 adds Whisper support

#18
post #4

Shut off the broken bot filter so we can read it please

Check out commit 13ce36fef98a3f4e6d8360c24d6b8434cbb8869b from https://git.ffmpeg.org/ffmpeg.git if your web browser doesn't support Javascript. The linked page is just a git viewer for that specific commit.

Re: FFmpeg 8.0 adds Whisper support

#20
post #8

Am I correct in understanding that Whisper is a speech recognition AI model originally created by OpenAI? https://en.wikipedia.org/wiki/Whisper_(speech_recognition_sy...

I think so, if I remember correctly PotPlayer also supports it for automatic subtitling.
Post reply on HN