Live data from Hacker News

High-performance speech recognition with no supervision at all

ai.facebook.com

51–60 of 66 posts

Re: High-performance speech recognition with no supervision at all

#51
post #16

Why would FB work on such a tech?

Are you looking at Facebook with glasses from 2010? Because one would expect a almost-trillion-dollar company to work on more than a single-focus product. Why would a search engine (1997) make glasses (2013)?

>Why would a search engine (1997) make glasses (2013)?

It literally was just because Sergey Brin thought it was cool.

Re: High-performance speech recognition with no supervision at all

#52
post #37

Earlier quoted context omitted.

Nope, like autonmous driving the problem will be the last 20%

Just because one overhyped tech is not working out does not mean all new tech will fail. We already have noise cancellation, good tiny batteries, fast wireless internet, and all the tech to learn spoken languages without supervision! Autonomous driving has a much higher bar for accuracy.

Are you sure? A translation error played part in the dropping of the atomic bomb on Hiroshima.

https://pangeanic.com/knowledge/the-worst-translation-mistak...

Language has great ambiguity, depends heavily on time period, context and tone.

https://www.reddit.com/r/funny/comments/2chfge/you_need_some... That could lead to problems even without translation.

Re: High-performance speech recognition with no supervision at all

#53
Question: are there any efforts to communally create / crowd source training material for neural networks?

I’m thinking language like this, but also labelled imagery for, for example, face detection, which works better on white people.

Has anyone attempted to create a way for people to create and donate labelled data to a dataset?

Re: High-performance speech recognition with no supervision at all

#54
post #52

Earlier quoted context omitted.

Just because one overhyped tech is not working out does not mean all new tech will fail. We already have noise cancellation, good tiny batteries, fast wireless internet, and all the tech to learn spoken languages without supervision! Autonomous driving has a much higher bar for accuracy.

Are you sure? A translation error played part in the dropping of the atomic bomb on Hiroshima. https://pangeanic.com/knowledge/the-worst-translation-mistak... Language has great ambiguity, depends heavily on time period, context and tone. https://www.reddit.com/r/funny/comments/2chfge/you_need_some... That could lead to problems even without translation.

So maybe don't use the tech yet in such high-stakes situations. In my personal life, an automated translation error is very unlikely to cause a serious problem, while an automated driving error could kill me.

(Fwiw, if you read the comments at your second link, you'll find the image is a fake.)

Re: High-performance speech recognition with no supervision at all

#55

Question: are there any efforts to communally create / crowd source training material for neural networks? I’m thinking language like this, but also labelled imagery for, for example, face detection, which works better on white people. Has anyone attempted to create a way for people to create and donate labelled data to a dataset?

This project is more on the academic side -- disclaimer, I was involved in it, but it's led by Justin Harris at Microsoft Research: "Sharing Updatable Models on Blockchain" https://github.com/microsoft/0xDeCA10B

The idea is that the smart contract is a learning algorithm, and people can donate data to a public repository stored on a blockchain. The learned model is publicly available for everyone. People can also receive incentives for donating data in some implementations.

Re: High-performance speech recognition with no supervision at all

#56
Could someone explain what is being done here?

I see they are using a GAN, and doing unsupervised training. But then they appear to compare their model to supervised-trained models.

How do they do this? Do they tack a supervised-trained model onto the end of their unsupervised model? I imagine they must do supervised training at some point, else how can they convert sounds to text?

Re: High-performance speech recognition with no supervision at all

#57
That's great, but it still performs 2 times worst than the best supervised model.

Also : "The discriminator itself is also a neural network. We train it by feeding it the output of the generator as well as showing it real text from various sources that were phonemized."

Is the "real text from various sources that were phonemized" a manually labelized database? If Yes, that step is supervised, which makes the whole thing actually supervised to some extent

Re: High-performance speech recognition with no supervision at all

#58
post #57

That's great, but it still performs 2 times worst than the best supervised model. Also : "The discriminator itself is also a neural network. We train it by feeding it the output of the generator as well as showing it real text from various sources that were phonemized." Is the "real text from various sources that were phonemized" a manually labelized database? If Yes, that step is supervised, which makes the whole th…

If phonemization is converting words to phonetic symbols, it could be taken from existing human-written dictionaries automatically.

Re: High-performance speech recognition with no supervision at all

#59
Was anyone else surprised that they used simple k-means for clustering the phonemes? I skimmed the paper and it looks like they use a really high k (128), presumably to avoid having to do an elbow-plot like approach.

Maybe the computational benefits of such a simple algorithm outweighed the potentially bad clusters. Thoughts? I'll try to read the paper in more depth later in case they explain the choice and I missed it.

Re: High-performance speech recognition with no supervision at all

#60
post #35

Earlier quoted context omitted.

Our main usages of speech rec are: - Understanding for videos for content moderation, ranking and recommendations - Captioning - Assistant tech in Portal, Oculus and upcoming glasses

For me the most scary part is not doing end-to-end encryption in Messenger, which means that the US government has a right to listen to my conversations on it.

Not if you use secret conversation mode or just use whatsapp
Post reply on HN