Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

Post reply on HN
Print view View original

141–150 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#141

Post by shrx »

pawelduda wrote: post #124
kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

Can you give an example why it made your life that much better?

As a hard of hearing person, I can now download any video from the internet (e.g. youtube) and generate subtitles on the fly, not having to struggle to understand badly recorded or unintelligible speech.

Re: FFmpeg 8.0 adds Whisper support

#142

Post by einpoklum »

ks2048 wrote: post #51
instagraham wrote: post #43

Does this mean that any software which uses ffmpeg can now add a transcription option? Audacity, Chrome, OBS etc

If they want to support it out-of-the box, they'll still have to embed a model file (roughly 500 MB - 3GB, varying size and quality)

Can't you point ffmpeg to a model file using some preferences dialog?

Re: FFmpeg 8.0 adds Whisper support

#143

Post by ctxc »

JohnKemeny wrote: post #99
anonymousiam wrote: post #96

Earlier quoted context omitted.

Whisper is excellent, but not perfect. I used Whisper last week to transcribe a phone call. In the transcript, the name of the person I was speaking with (Gem) was alternately transcribed as either "Jim" or "Jem", but never "Gem."

Whisper supports adding a context, and if you're transcribing a phone call, you should probably add "Transcribe this phone call with Gem" , in which case it would probably transcribe more correctly.

Thanks John Key Many!

Re: FFmpeg 8.0 adds Whisper support

#144

Post by [deleted] »

kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

[deleted]

Re: FFmpeg 8.0 adds Whisper support

#145

Post by vevoe »

abdusco wrote: post #117
MaxikCZ wrote: post #114

I tried to use whisper to generate non-english subs from english audio, but wasnt able to figure out. I know it can do english subs from non-english audio, and that earlier (less precise) versions could do any language audio -> any language subs, but latest whisper only to english subs. Anyone found a way?

I solved it by generating English subtitles, then passing those to an LLM in chunks that are ~20 entries in size. Include preceding and following subtitles as context for better translation. Make sure to replace the timestamps with simple integer ids, because LLMs like to mangle those, no matter how hard you prompt. I could share a python script that is working pretty reliably for me.

I'd love to see that script, do you have a link?

Re: FFmpeg 8.0 adds Whisper support

#146

Post by iAMkenough »

ambicapter wrote: post #87
Lio wrote: post #45

Once local transcription is in more places hopefully we can persuade content creator not to burn bouncing sub-titles into their videos. I've seen professionally produced recordings on dry and technical subjects with good sound quality where they've decided to use distracting sub-titles with no way to disable them. It seems so unnecessary if you're not making novelty videos about cats. Also local transcription allows…

They do that because it increases “engagement”, not because they care about the user’s experience with the subtitles.

Also some social media platforms don't offer subtitle functionality, so burned-in is the only way if you want to serve your content to people that require subtitles or refuse to unmute their phones while they watch from their toilet.

Re: FFmpeg 8.0 adds Whisper support

#147

Post by mmmpetrichor »

diggan wrote: post #6
zzsshh wrote: post #3

Does this finally enable dynamically generating subtitles for movies with AI?

Finally? I think VLC demo'd this a while ago at some conference where they had a table, if I remember correctly.

I've been waiting a while now for automatic translated subtitles in vlc. I thought it would be here by now. I'm probably underestimating the difficulty but I'm surprised some video player hasn't done it by now. (as far as I know).

Re: FFmpeg 8.0 adds Whisper support

#148

Post by SSLy »

demurgos wrote: post #125
SSLy wrote: post #13

Earlier quoted context omitted.

VLC and ffmpeg are unrelated projects

I'm not very familiar with them, but I always assumed that there is a lot of overlap between the maintainers of both projects.

Well, they are just unrelated. VLC has a plugin to access ffmpeg codecs via libav*, that's about it.

Re: FFmpeg 8.0 adds Whisper support

#149

Post by kmfrk »

pawelduda wrote: post #124
kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

Can you give an example why it made your life that much better?

Aside from accessibility as mentioned, you can catch up on videos that are hours long. Orders of magnitude faster than watching on 3-4x playback speed. If you catch up through something like Subtitle Edit, you can also click on relevant parts of the transcript and replay it.

But transcribing and passably translating everything goes a long way too. Even if you can hear what's being said, it's still less straining to hear when there's captions for it.

Obviously one important factor to the convenience is how fast your computer is at transcription or translation. I don't use the features in real-time personally currently, although I'd like to if a great UX comes along through other software.

There's also a great podcast app opportunity here I hope someone seizes.

Re: FFmpeg 8.0 adds Whisper support

#150

Post by taminka »

kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

whisper is great, i wonder why youtube's auto generated subs are still so bad? even the smallest whisper is way better than google's solution? is it licensing issue? harder to deploy at scale?
Post reply on HN