Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

191–200 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#191

Earlier quoted context omitted.

Thanks. Now I know that I'm not that stupid and this actually makes no sense

It actually does make sense. Not saying you're stupid, but in standard English, if you say it quickly, the two sentences are nearly identical.

But in "you sing", "s" is pronounced as "s", not as "z" from "using", right?

Re: FFmpeg 8.0 adds Whisper support

#192

Why would one use FFmpeg with Whisper support, instead of using Whisper directly?

I run a service that does transcriptions as part of the pipeline, and I use ffmpeg for other parts (such as speeding up audio). Having it all on a single command might make sense for some people if the costs work out.

Re: FFmpeg 8.0 adds Whisper support

#193
post #69

I know nothing about Whisper, is this usable for automated translation? I own a couple very old and as far as I'm aware never translated Japanese movies. I don't speak Japanese but I'd love to watch them. A couple years ago I had been negotiating with a guy on Fiver to translate them. At his usual rate-per-minute of footage it would have cost thousands of dollars but I'd negotiated him down to a couple hundred before…

In my experience it works ok. The "English" model actually knows a lot of languages and will translate directly to English.

You can also transcribe it to Japanese and use a translator to convert to English. This can sometimes help for more semantically complex dialogue.

For example, using faster-whisper-xxl [1]:

Direct translation:

    faster-whisper-xxl.exe --language English --model large-v2 --ff_vocal_extract mdx_kim2 --vad_method pyannote_v3 --standard 
Use Japanese, then translate:

    faster-whisper-xxl.exe --language Japanese --task translate --model large-v2 --ff_vocal_extract mdx_kim2 --vad_method pyannote_v3 --standard 
1. https://github.com/Purfview/whisper-standalone-win

Re: FFmpeg 8.0 adds Whisper support

#194
post #154

Earlier quoted context omitted.

A slight segue to this; I was made aware of the phenomena that - The language in which you think in, sets the constraints to which you level of expanse the brain can think and parse information in. I think in English fortunately and it's an ever evolving language so, expanding as the world does. That is compared to the majority of people where I'm from; English was a second language they had to learn and the people t…

This is called linguist relativity (nee. The Sapir-Whorf hypothesis) and the strong form you describe has fallen out of favour in modern linguistics. A surprising number of monolingual people think their own language is the most adaptable and modern language, but this is obviously untrue. All languages evolve to fit the needs of speakers. Also, the idea that people "think in language X" is heavily disputed. One obvio…

My experience is that sometimes, for example, when I watch a lecture in a foreign language, there could be some terms for which I don't know the correct translation so I cannot think about or mention them in my native language, while I understand what they mean.

Re: FFmpeg 8.0 adds Whisper support

#197
post #192

Why would one use FFmpeg with Whisper support, instead of using Whisper directly?

I run a service that does transcriptions as part of the pipeline, and I use ffmpeg for other parts (such as speeding up audio). Having it all on a single command might make sense for some people if the costs work out.

Terrific, thank you.

Re: FFmpeg 8.0 adds Whisper support

#198

Earlier quoted context omitted.

A good opportunity to point people to the paper with my favorite title of all time: "How to wreck a nice beach you sing calm incense" https://dl.acm.org/doi/10.1145/1040830.1040898

Do AI voice recognition still use markov models for this?

Whisper uses an encoder-decoder transformer.

Re: FFmpeg 8.0 adds Whisper support

#200
post #69

I know nothing about Whisper, is this usable for automated translation? I own a couple very old and as far as I'm aware never translated Japanese movies. I don't speak Japanese but I'd love to watch them. A couple years ago I had been negotiating with a guy on Fiver to translate them. At his usual rate-per-minute of footage it would have cost thousands of dollars but I'd negotiated him down to a couple hundred before…

Hey, indeed Whisper can do the transcription of Japanese and even the translation (but only to English). For the best results you need to use the largest model which depending on your hardware might be slow or fast.

Another option is to use something like VideoToTextAI which allows you to transcribe it fast and then translate it into 100+ languages which you can then export the subtitle (SRT) file for

Post reply on HN