Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

github.com

Post reply on HN
Print view View original

131–140 of 354 posts

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#131

Post by bmacho »

pferde wrote: post #81
wahnfrieden wrote: post #73

Earlier quoted context omitted.

whisper MUST be combined with silence detection / VAD

Ah, the good old "you're holding it wrong". What good is a speech recognition tool that literally hears imaginary voices?

> What good is a speech recognition tool that literally hears imaginary voices?

Well, if it is supposed to work after silence detection, then it is good for speech recognition I guess. It's like blaming a wheel why is it circular, you can't sit on it. It's a part of a larger machine.

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#132

Post by bmacho »

baobabKoodaa wrote: post #52
wongarsu wrote: post #48

Earlier quoted context omitted.

And the German is “subtitles of [public broadcaster] for [content network], 2017 I'm not sure this is really overfitting, the network does exactly what the training data demands. According to the training data silence art the end transcribes to a copyright notice or subtitle credits

> I'm not sure this is really overfitting, the network does exactly what the training data demands. What do you think overfitting is, if not that?

Overfitting is achieving better and better scores on the training material and worse and worse scores on unseen tasks. More at: https://en.wikipedia.org/wiki/Overfitting#Machine_learning

This is just wrong training data.

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#133

Post by cyp0633 »

ttflee wrote: post #99
cyp0633 wrote: post #24

The same happens with whisper-large-v3 on Chinese transcription: silence is transcribed to something like "please upvote, share and favourite this video". I suspect they trained the model on some random YouTube video without carefully picking really useful data.

In Chinese, it always added something like "For study/research purpose only. Please delete after 48 hours." This is what those volunteers added in subtitles of (pirated) movies/shows.

That is not the case here - I never encountered this with whisper-large-v3 or similar ASR models. Part of the reason, I guess, is that those subs are burnt into the movie, which makes them hard to extract. Standalone subs need the corresponding video resource to match the audio and text. So nothing is better than YouTube videos which are already aligned.

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#134

Post by Bluestein »

msopena wrote: post #101
o1bf2k25n8g5 wrote: post #98

Earlier quoted context omitted.

I have discovered a truly marvelous proof of how to smash that like and subscribe button, which this comment box is too small to contain.

Signed by Pierre de FermAIt

NaN

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#135

Post by voidUpdate »

berkes wrote: post #87
flexagoon wrote: post #47

In Russian it often hallucinates "Субтитры сделал DimaTorzok" ("Subtitles by DimaTorzok") at the end of things. Interestingly, I wasn't able to find any YouTube videos with that name in the subtitles, so it's not like it's in a lot of training data.

Could it be someone distributing subs online, e.g. showing up in the opensubtitles.org dataset?

Or possibly someone subtitling pirated movies? That seems to be a common thing according to other comments

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#136

Post by scotty79 »

Incipient wrote: post #31
actionfromafar wrote: post #28

Earlier quoted context omitted.

Haha, trained on torrented movies! :-D The MPA must be so proud.

It's absolutely insane that these companies can't be held liable for what is obvious piracy.

What's insane is copyright. How come you can own intellectual property but not pay a property tax? The ecosystem would be much healthier if to get copyright protections you should declare value of your IP (that you are obligated to sell for if the buyer pops up) and pay tax on this for every year you hold the IP.

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#137

Post by stndef »

indrora wrote: post #130
cyp0633 wrote: post #24

The same happens with whisper-large-v3 on Chinese transcription: silence is transcribed to something like "please upvote, share and favourite this video". I suspect they trained the model on some random YouTube video without carefully picking really useful data.

When YouTube began building automatic transcriptions for captions, it regularly flagged any noise or music -- typically industrial noise -- with "[foreign]" If it couldn't understand it, it was "foreign" for the longest time.

Yeah, I can confirm seeing that a fair bit specifically during non-verbal parts of videos when someone is using a tool.

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#138

Post by heavyset_go »

stingraycharles wrote: post #97
bGl2YW5j wrote: post #60

Earlier quoted context omitted.

If the model was able to generalise, you’d expect it to output something like “[silence]” or “…”, in response to silence. Instead, it reverted to what it has seen before (in the training data), hence the overfit.

Right, maybe my definition of overfitting was wrong, I always understood it more as trying to optimize for a specific benchmark / use case, and then it starts failing in other areas. But the way you phrase it, it’s just “the model is not properly able to generalize”, ie it doesn’t understand the concept of silence also makes sense. But couldn’t you then argue that any type of mistake / unknown could be explained as “…

Your definition is one, but the one the OP is using is overfitting to training data.

Re: Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic

#140

Post by Hnrobert42 »

nottorp wrote: post #58

Title should be changed to "OpenAI publishes evidence they trained on pirated movies".

HN is pretty strict about not editorializing titles. Even if you statement was unequivocably correct, the post would get flagged.
Post reply on HN