Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

github.com

Post reply on HN
Print view View original

141–150 of 354 posts

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#141

Post by ilyakaminsky »

dandiep wrote: post #4

Whisper is unusable IMO because of the hallucinations. Widely documented. Removing silence from audio clips helps, but even then it will auto correct grammar, translating bilingual speech, etc. Improved in the latest audio models but not solved [1] 1. https://news.ycombinator.com/item?id=43427376

I wouldn't describe it as "unusable" so much as needing to understand its constraints and how to work around them. I built a business on top of Whisper [1] and one of the early key insights was to implement a good voice activity detection (VAD) model in order to reduce Whisper's hallucinations on silence.

[1] https://speechischeap.com

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#144

Post by wahnfrieden »

pferde wrote: post #81
wahnfrieden wrote: post #73

Earlier quoted context omitted.

whisper MUST be combined with silence detection / VAD

Ah, the good old "you're holding it wrong". What good is a speech recognition tool that literally hears imaginary voices?

Yes, you are holding it wrong. The good of it is that it does not output imaginary voices when used with VAD.

Show us a technology with better results that does not use VAD. If you can’t, then I’m not sure what you’re arguing against except superficialities so inconsequential that I can’t comprehend the condescension. The results speak for itself

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#145

Post by wahnfrieden »

Xmd5a wrote: post #92
pferde wrote: post #81

Earlier quoted context omitted.

Ah, the good old "you're holding it wrong". What good is a speech recognition tool that literally hears imaginary voices?

faster-whisper has a min_silence_duration_ms option

There are much higher quality VAD solutions available

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#146

Post by retsibsi »

scotty79 wrote: post #136
Incipient wrote: post #31

Earlier quoted context omitted.

It's absolutely insane that these companies can't be held liable for what is obvious piracy.

What's insane is copyright. How come you can own intellectual property but not pay a property tax? The ecosystem would be much healthier if to get copyright protections you should declare value of your IP (that you are obligated to sell for if the buyer pops up) and pay tax on this for every year you hold the IP.

> if to get copyright protections you should declare value of your IP (that you are obligated to sell for if the buyer pops up) and pay tax on this for every year you hold the IP

I think this would have some unpalatable consequences. Let's say an author is writing a modestly successful book series: it's not going to make them rich, but it's commercially viable and they care a lot about it for its own sake. Under this system, if the author declares a value commensurate with the (quite small) pure economic value of the IP, they have to live in fear of their right to continue working on their creation being abruptly taken away from them at any point. If they instead declare a value commensurate with the economic value + the extra value that it has to them personally, the resulting tax liability could easily tip the balance and destroy their ability to pursue their writing as a career.

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#147

Post by jovial_cavalier »

Looks like it's some random user who has generated some lyricslyrics translations between Arabic and English. It's strange, they don't seem to have many contributions. I would have imagined them to be more prolific.

https://lyricstranslate.com/en/translator/nancy-qunqar

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#148

Post by probably_wrong »

mmcwilliams wrote: post #128
cyp0633 wrote: post #24

The same happens with whisper-large-v3 on Chinese transcription: silence is transcribed to something like "please upvote, share and favourite this video". I suspect they trained the model on some random YouTube video without carefully picking really useful data.

Similar in the English model. Pretty clear they trained on YouTube videos where creators will put that in otherwise silent sections to ensure it shows up for people with CC on.

The number one hallucination in my transcriptions was "Subtitles by the Amara.org community".

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#149

Post by stingraycharles »

heavyset_go wrote: post #138
stingraycharles wrote: post #97

Earlier quoted context omitted.

Right, maybe my definition of overfitting was wrong, I always understood it more as trying to optimize for a specific benchmark / use case, and then it starts failing in other areas. But the way you phrase it, it’s just “the model is not properly able to generalize”, ie it doesn’t understand the concept of silence also makes sense. But couldn’t you then argue that any type of mistake / unknown could be explained as “…

Your definition is one, but the one the OP is using is overfitting to training data.

That’s exactly my point: by that definition any incorrect answer can be explained by “overfitting to training data”.

Where do you draw the line between “overfitting to training data” and “incorrect data” ?

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#150

Post by xandrius »

pferde wrote: post #81
wahnfrieden wrote: post #73

Earlier quoted context omitted.

whisper MUST be combined with silence detection / VAD

Ah, the good old "you're holding it wrong". What good is a speech recognition tool that literally hears imaginary voices?

So if a tool has a process to have it perform at its best then it's a problem?

Do you also moan that before applying glue to a surface or it won't stick? Or if you need to drill a guiding hole before making a larger one in wood? Or that you need to use truly prime numbers for a security key to actually be safe?

Post reply on HN