Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg 8.0 adds Whisper support

code.ffmpeg.org

Post reply on HN
Print view View original

321–330 of 341 posts

Re: FFmpeg 8.0 adds Whisper support

#321

Post by ddingus »

spauldo wrote: post #302
gregoryl wrote: post #290

Earlier quoted context omitted.

I'd love your thoughts on why it might be hardware. I reason that my hearing is generally fine - there's no issue picking apart loud complex music (I love breakcore!). But play two songs at the same time, or try talking to me with significant background noise, and I seem to be distinctly impaired vs. most others. If I concentrate, I can sometimes work through it. My uninformed model is a pipeline of sorts, and some s…

I don't have much beyond what I said. It happened to me after repeated exposure to dangerously loud sounds in a small room. I can hear faint sounds, but I have trouble with strong accents and I can't understand words if there's a lot of background noise. I noticed it shortly after I left that band, and I left because the last practice was so loud it felt like a drill boring into my ears. I don't think I have any hard…

It seems to me your ability to discriminate has been impacted.

I have always pictured it working this way:

In the Cochlea, we have all the fine hair like sensors. The spread of them determines our range of frequencies, and this declines with age. Usually not too much, but could be as much as half. 10 to 12khz.

Good news in that is all the good stuff we crave is below 10khz. Don't sweat age related hearing loss too much.

The number of these sensors determines our ability to hear concurrent sounds, or complexity.

The shape of them impacts how loud sounds need to be to be heard.

Chances are, your loud exposure had harmonics that impacted many of these sensing hairs, but not in one place. The result is a loss of discrimination of concurrent sounds.

There are plenty to cover the frequency range, so things do not seem muffled or low. Their shape is good, not worn so you hear faint sounds well.

The lower number of them is the issue. Or, they are still there, just bent-- something prevents them from contrubuting.

Another way to think of this is in reverse:

Say you had 30 oscillators you could start at any frequency and time. How complex of a sound could you make? Now cut that in half.

What is lost?

The most complex, concurrent sound cases.

Re: FFmpeg 8.0 adds Whisper support

#322

Post by crazygringo »

nomdep wrote: post #300
crazygringo wrote: post #240

Earlier quoted context omitted.

I mean, subtitles are mostly the same. Sometimes they're edited down simply for space, because there wouldn't be time to easily read all the dialog otherwise. And sometimes repetition of words or phrases is removed, because it's clearer, and the emphasis is obvious from watching the moving image. And filler words like "uh" or "um" generally aren't included unless they were in the original script. Most interestingly,…

Hard and vehemently disagree. Subtitles are not commentary tracks. The artists are the writers, voice actors, and everyone else involved in creating the original media. Never, ever, a random stranger should contaminate it with his/her opinions or point of views. Subtitles should be perfect transcriptions or the most accurate translations, never reinterpretations

Nobody said subtitles are commentary tracks.

And official subtitles aren't made by random strangers. They're made by people who do it professionally.

It's not "contamination" or "opinions", like somebody is injecting political views! And certainly not "reinterpretation". Goodness. It's about clarity, that's all.

Also there's no such thing as the "most accurate" translations. Translations themselves are an art, hugely.

Re: FFmpeg 8.0 adds Whisper support

#323

Post by randomflyer20 »

joshvm wrote: post #265
pawelduda wrote: post #124

Earlier quoted context omitted.

Can you give an example why it made your life that much better?

I don't know about much better, but I like Whisper's ability to subtitle foreign language content on YouTube that (somehow) doesn't have auto-generated subs. For example some relatively obscure comedy sketches from Germany where I'm not quite fluent enough to go by ear. 10 years ago you'd be searching through random databases to see if someone had synchronized subtitles for the exact copy of the video that you had. O…

this is similar to what you are saying: https://x.com/thekrishdesai/status/1955390536422134109

Re: FFmpeg 8.0 adds Whisper support

#324

Post by troad »

solardev wrote: post #274
strken wrote: post #134

Earlier quoted context omitted.

Thank you! "Calm incense" makes very little sense when said in an accent where calm isn't pronounced like com.

How is calm pronounced in those accents?

It's not 'calm' that differs, it's 'common'. Calm like palm, in all major accents.

Traditionally, calm and com- have different vowels in English, but most North American accents merge com- into calm. All other major English accents retain the distinction.

If you're American, try saying 'com' while rounding your lips. Or just listen to a recording of 'common' in an online dictionary from Britain or Australia. (Or lot, pot, spot, etc.)

TLDR (simplified):

US/Ca: (lot = palm) ≠ start

UK/Au: lot ≠ (palm = start)

Re: FFmpeg 8.0 adds Whisper support

#326

Post by Sophira »

yvdriess wrote: post #94
londons_explore wrote: post #23

Does this have the ability to edit historic words as more info becomes available? Eg. If I say "I scream", it sounds phonetically identical to "Ice cream". Yet the transcription of "I scream is the best dessert" makes a lot less sense than "Ice cream is the best dessert". Doing this seems necessary to have both low latency and high accuracy, and things like transcription on android do that and you can see the adjusti…

A good opportunity to point people to the paper with my favorite title of all time: "How to wreck a nice beach you sing calm incense" https://dl.acm.org/doi/10.1145/1040830.1040898

Fun fact, I just could not work out what this was supposed to be, so I just used Whisper (indirectly, via the FUTO Voice Input app on my phone) and repeated the sentence into it, and it came out with the 'correct' transcription of "How to recognize speech using common sense." first time.

Of course, this is nothing like what I actually said, so... make your own mind up whether that is actually a correct transcription or not!

I have a British accent, for the record.

Re: FFmpeg 8.0 adds Whisper support

#327

Post by numpad0 »

okwhateverdude wrote: post #319
abdullahkhalids wrote: post #226

Earlier quoted context omitted.

In South Asia, it's quite common for people to speak a combination of their local language and English. Not just alternating sentences between the two languages, but in fact, constructing sentences using compound phrases from the two languages. "Madam, please believe me, maine homework kiya ha" [I did my homework].

This is common in the southwestern part of the US too. My partner and her friends she grew up with will have conversations that fluidly pick phrases and vocab from either Spanish or English depending on what words happen to be the easiest to pull from their brain. It's wild to listen to.

Aren't those limited to specific words or phrases in specific forms? I doubt it works for arbitrary half-sentences.

Re: FFmpeg 8.0 adds Whisper support

#329

Post by spagettnet »

notatallshaw wrote: post #140
kmfrk wrote: post #93

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video >…

> uv pip install --system torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 uv has a feature to get the correct version of torch based on your available cuda (and some non-cuda) drivers (though I suggest using a venv not the system Python): > uv pip install torch torchvision torchaudio --torch-backend=auto More details: https://docs.astral.sh/uv/guides/integration/pytorch/#automa... This…

Of all the great things people say about UV, this is the one that sold me on it when I found this option in the docs. Such a nice feature.
Post reply on HN