Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

Post reply on HN
Print view View original

101–110 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#101

Post by no_wizard »

londons_explore wrote: post #48
jeroenhd wrote: post #37

Earlier quoted context omitted.

The ffmpeg code seems to default to three second chunks ( https://ffmpeg.org/ffmpeg-filters.html#whisper-1 ): queue The maximum size that will be queued into the filter before processing the audio with whisper. Using a small value the audio stream will be processed more often, but the transcription quality will be lower and the required processing power will be higher. Using a large value (e.g. 10-20s) will produce m…

so if "I scream" is in one chunk, and "is the best dessert" is in the next, then there is no way to edit the first chunk to correct the mistake? That seems... suboptimal! I don't think other streaming transcription services have this issue since, whilst they do chunk up the input, past chunks can still be edited. They tend to use "best of N" decoding, so there are always N possible outputs, each with a probability as…

That’s because at the end of the day this technology doesn’t “think”. It simply holds context until the next thing without regard for the previous information

Re: FFmpeg 8.0 adds Whisper support

#103

Post by tossit444 »

kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

Aegisub is still actively developed (forked), and imo, both software can't really be compared to one another. They can complement each other, but SE is much better for actual transcription. Aegisub still does the heavy lifting for typesetting and the like.

Re: FFmpeg 8.0 adds Whisper support

#104

Post by webinar »

I've been using FFmpeg and Whisper to record and transcribe live police scanner audio for my city, and update it in real-time to a live website. It works great, with the expected transcription errors and hallucinations.

Re: FFmpeg 8.0 adds Whisper support

#105

Post by whywhywhywhy »

Lio wrote: post #45

Once local transcription is in more places hopefully we can persuade content creator not to burn bouncing sub-titles into their videos. I've seen professionally produced recordings on dry and technical subjects with good sound quality where they've decided to use distracting sub-titles with no way to disable them. It seems so unnecessary if you're not making novelty videos about cats. Also local transcription allows…

Algorithm boosts it that’s why they do it. Even if every device had real time 100% accurate subtitling built in they’d still do it if they video performs better with it.

Re: FFmpeg 8.0 adds Whisper support

#106

Post by abound »

yvdriess wrote: post #94
londons_explore wrote: post #23

Does this have the ability to edit historic words as more info becomes available? Eg. If I say "I scream", it sounds phonetically identical to "Ice cream". Yet the transcription of "I scream is the best dessert" makes a lot less sense than "Ice cream is the best dessert". Doing this seems necessary to have both low latency and high accuracy, and things like transcription on android do that and you can see the adjusti…

A good opportunity to point people to the paper with my favorite title of all time: "How to wreck a nice beach you sing calm incense" https://dl.acm.org/doi/10.1145/1040830.1040898

For folks like me puzzling over what the correct transcription of the title should be, I think it's "How to recognize speech using common sense"

Re: FFmpeg 8.0 adds Whisper support

#107

Post by Xunjin »

webinar wrote: post #104

I've been using FFmpeg and Whisper to record and transcribe live police scanner audio for my city, and update it in real-time to a live website. It works great, with the expected transcription errors and hallucinations.

Is this website open? Would love to see your work :P

Re: FFmpeg 8.0 adds Whisper support

#108

Post by fmx »

yvdriess wrote: post #94
londons_explore wrote: post #23

Does this have the ability to edit historic words as more info becomes available? Eg. If I say "I scream", it sounds phonetically identical to "Ice cream". Yet the transcription of "I scream is the best dessert" makes a lot less sense than "Ice cream is the best dessert". Doing this seems necessary to have both low latency and high accuracy, and things like transcription on android do that and you can see the adjusti…

A good opportunity to point people to the paper with my favorite title of all time: "How to wreck a nice beach you sing calm incense" https://dl.acm.org/doi/10.1145/1040830.1040898

The paper: https://sci-hub.st/https://dl.acm.org/doi/10.1145/1040830.10...

(Agree that the title is awesome, by the way!)

Re: FFmpeg 8.0 adds Whisper support

#110

Post by Fluorescence »

londons_explore wrote: post #23

Does this have the ability to edit historic words as more info becomes available? Eg. If I say "I scream", it sounds phonetically identical to "Ice cream". Yet the transcription of "I scream is the best dessert" makes a lot less sense than "Ice cream is the best dessert". Doing this seems necessary to have both low latency and high accuracy, and things like transcription on android do that and you can see the adjusti…

It makes me curious about how human subtitlers or even scriptwriters choose to transcribe intentionally ambiguous speech, puns and narratively important mishearings. It's like you need to subtitle what is heard not what is said.

Do those born profoundly deaf specifically study word sounds in order to understand/create puns, rhymes and such so they don't need assistance understanding narrative mishearings?

It must feel like a form of abstract mathematics without the experiential component... but then I suspect mathematicians manufacture an experiential phenomena with their abstractions with their claims of a beauty like music... hmm!

Post reply on HN