Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

Post reply on HN
Print view View original

251–260 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#251

Post by slhck »

manca wrote: post #249

The only problem with this PR/diff is that it creates just a avfilter wrapper around whisper.cpp library and requires the user to manage the dependencies on their own. This is not helpful for novice users who will first need to: 1. git clone whisper.cpp 2. Make sure they have all dependencies for `that` library 3. Hope the build passes 4. Download the actual model AND only then be able to use `-af "whisper=model...`…

While that would be nicer from an end-user perspective, it's something hard to maintain for FFmpeg itself. Consider the velocity of the whisper-cpp project. I'm sure that – just like with filters such as vmaf, which also require building a dependency and downloading a model – precompiled versions will become available for novice users to directly download. Especially considering whisper-cpp is MIT-licensed.

Re: FFmpeg 8.0 adds Whisper support

#252

Post by Morizero »

kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

You don't happen to know a whisper solution that combines diarization with live audio transcription, do you?

Re: FFmpeg 8.0 adds Whisper support

#253

Post by numpad0 »

smallpipe wrote: post #181
0cf8612b2e1e wrote: post #174

Earlier quoted context omitted.

The quality of subtitles implies that almost no effort is being put into their creation. Watch even a high budget movie/TV show and be aghast at how frequently they diverge.

A good subtitle isn't a perfect copy of what was said.

Aren't same-language subtitles supposed to be perfect literal transcripts, while cross-language subtitling is supposed to be compressed creative interpretations?

Re: FFmpeg 8.0 adds Whisper support

#255

Post by sigbottle »

cyphar wrote: post #154
mockingloris wrote: post #67

Earlier quoted context omitted.

A slight segue to this; I was made aware of the phenomena that - The language in which you think in, sets the constraints to which you level of expanse the brain can think and parse information in. I think in English fortunately and it's an ever evolving language so, expanding as the world does. That is compared to the majority of people where I'm from; English was a second language they had to learn and the people t…

This is called linguist relativity (nee. The Sapir-Whorf hypothesis) and the strong form you describe has fallen out of favour in modern linguistics. A surprising number of monolingual people think their own language is the most adaptable and modern language, but this is obviously untrue. All languages evolve to fit the needs of speakers. Also, the idea that people "think in language X" is heavily disputed. One obvio…

I think it's more like, you have a thought X, that has so many dimensions to it, but the way you serialize it to something that's actually discussable and comparable to other thoughts is language. And sometimes that language naturally loves slicing one part of that thought one way or the other.

(then there's also a feedback loop type of argument, that always happens when discussing any sort of perception-reality distinction, but let's ignore that for now)

At least for me, my brain is so bad and it's hard for me to truly hold a single thought in my head for a long time. Maybe it eventually settles into my subconscious but I don't really have a way to verify that.

Re: FFmpeg 8.0 adds Whisper support

#256

Post by kmfrk »

Morizero wrote: post #252
kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

You don't happen to know a whisper solution that combines diarization with live audio transcription, do you?

Proper diarization still remains a white whale for me, unfortunately.

Last I looked into it, the main options required API access to external services, which put me off. I think it was pyannotate.audio[1].

[1]: https://github.com/pyannote/pyannote-audio

Re: FFmpeg 8.0 adds Whisper support

#257

Post by iczero »

creesch wrote: post #241
kstrauser wrote: post #207

Earlier quoted context omitted.

Hard disagree. When I'm reading a transcript, I want word-for-word what the people said, not a creative edit. I want the speakers' voice, not the transcriptionist's. And when I'm watching subtitles in my own language (say because I want the volume low so I'm not disturbing others), I hate when the words I see don't match the words I hear. It's the quickest way I can imagine to get sucked out of the content and into a…

> When I'm reading a transcript That's the thing though, subtitles aren't intended as full transcripts . They are intended to allow a wide variety of people to follow the content. A lot of people read slower than they would hear speech. So subtitles often need to condense or rephrase speech to keep pace with the video. The goal is usually to convey meaning clearly within the time available on screen. Not to capture e…

I regularly enable YouTube subtitles. Almost always, they are a 100% verbatim transcription, excluding errors from auto-transcription. I am not annoyed in the slightest, and in fact I very much prefer that they are verbatim.

If you are too slow at reading subtitles, you can either slow down the video or train yourself to read faster. Or you can just disable the subtitles.

Re: FFmpeg 8.0 adds Whisper support

#259

Post by porridgeraisin »

dotancohen wrote: post #238
porridgeraisin wrote: post #229

Earlier quoted context omitted.

The LLM turns my unstructured command into structured command (a limited set of commands hardcoded in the prompt) and a script takes that and executes it. I have it do stuff like interact with google keep/google calendar using the CLI. Those are the most used actions but there's a few others . Of course all actions can be scheduled. The LLM can screw up now and then and output absolute garbage. But I've got a knack n…

That is fascinating, thank you very much for sharing. Good luck with the grad work.

Thank you:)
Post reply on HN