Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

291–300 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#291

Now if it only did separate speaker identification (diarization)

It’s fairly easy to get diarizarion working with pyannote.audio and https://huggingface.co/pyannote/speaker-diarization-3.1 with ffmpeg converting the audio first to 16kHz mono WAV file but it really depends a lot on the audio - two person podcast where the speakers allow each other space works but lots of people with overlapping voices on the audio - not so great

Re: FFmpeg 8.0 adds Whisper support

#292
post #163

Earlier quoted context omitted.

> uv pip install --system torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 uv has a feature to get the correct version of torch based on your available cuda (and some non-cuda) drivers (though I suggest using a venv not the system Python): > uv pip install torch torchvision torchaudio --torch-backend=auto More details: https://docs.astral.sh/uv/guides/integration/pytorch/#automa... This…

I love uv and really feel like I only need to know "uv add" and "uv sync" to be effective using it with python. That's an incredible feat. But, when I hear about these kinds of extras, it makes me even more excited. Getting cuda and torch to work together is something I have struggled countless times. The team at Astral should be nominated for a Nobel Peace Prize.

Agreed, making the virtual environment management and so much else disappear lets so much more focus go to python itself.

Re: FFmpeg 8.0 adds Whisper support

#293

Earlier quoted context omitted.

I think the Dutch/English is probably the worst combination for this. Languages are rather close.

I don't understand how this would happen, though. It's not like it will mishear a dutch sentence as if it's english; it will correctly pick up the dutch sentence, but (since the language is auto-detected as english at the start of the segment), seemingly auto-translate that (correct and correctly heard) dutch text to english. All we need is a way to get the dutch text that's surely somewhere in there, before the tran…

Maybe try the turbo model which is transcription only. The other models were trained on x to en translations and they seem to emphasise the output language over the task token. You can get them to translate to any language even though it was never trained for that, comparatively nl-en translation is in the dataset so I'm not surprised it's doing that.

Re: FFmpeg 8.0 adds Whisper support

#294
post #285

Earlier quoted context omitted.

> I have that issue as well You say issue, I say feature. It's a great way to just ignore boring babbling at parties or other social engagements where you're just not that engaged. Sort of like selective hearing in relationships, but used on a wider audience

I don’t mean to speak for OP, but it strikes me as rude to make light of someone’s disability in this way. I’d guess it has caused them a lot of frustration.

Your assumption leads you to believe that I do not also suffer from the same issue. Ever since I was in a t-bone accident and the side airbag went off right next to my head, I have a definite issue hearing voices in crowded and noisy rooms with poor sound insulation. Some rooms are much worse than others.

So when I say I call it a feature, it's something I actually deal with unlike your uncharitable assumption.

Re: FFmpeg 8.0 adds Whisper support

#295

Earlier quoted context omitted.

Those burned in subtitles still aren’t as cool as theme-matched anime subtitles during intro music sequences from fansubs 15 years ago. Those are still cool IMO

Or how the fansubbers will create masks to translate diegetic text like signage and written notes

also love when a fansubber will just outright give you an asterisk explaining a joke that relies on nuance or wordplay

Re: FFmpeg 8.0 adds Whisper support

#297

I was expecting a lot more comments on if this is a necessary feature or if this even belongs in a library like ffmpeg. I think this is bloat, especially when the feature doesn't work flawless, whisper is very limited.

You'd be surprised what's in there, a few forms of NNs are already supported for denoising, speech detection.

I think having this flow out to all of the deps of libav is a greater good than notions of lib purity.

Re: FFmpeg 8.0 adds Whisper support

#300

Earlier quoted context omitted.

Hard disagree. When I'm reading a transcript, I want word-for-word what the people said, not a creative edit. I want the speakers' voice, not the transcriptionist's. And when I'm watching subtitles in my own language (say because I want the volume low so I'm not disturbing others), I hate when the words I see don't match the words I hear. It's the quickest way I can imagine to get sucked out of the content and into a…

I mean, subtitles are mostly the same. Sometimes they're edited down simply for space, because there wouldn't be time to easily read all the dialog otherwise. And sometimes repetition of words or phrases is removed, because it's clearer, and the emphasis is obvious from watching the moving image. And filler words like "uh" or "um" generally aren't included unless they were in the original script. Most interestingly,…

Hard and vehemently disagree. Subtitles are not commentary tracks.

The artists are the writers, voice actors, and everyone else involved in creating the original media. Never, ever, a random stranger should contaminate it with his/her opinions or point of views.

Subtitles should be perfect transcriptions or the most accurate translations, never reinterpretations

Post reply on HN