Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why don't we use subtitled films/tv to train speech recognition?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 40 posts

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#31

Post by etcet »

amirmc wrote: post #16
tcarnell wrote: post #11

I used to work for a company that built Speech Recognition systems and I came up with a similar/related idea - the idea being to take a load of videos of barack obama (for example), and create an accurate 'voice print'. Once done, any videos or speech could be scanned and if Barack Obama's voice print was recognized/detected, the recognizer could be tuned to his voice print AND could apply a set of appropriate gramma…

> when you google for text, you could actually retrieve videos whose content exactly matches the search terms and jump directly to that part of the video The search aspect of this is very interesting and I hadn't thought of it before (though in hindsight it seems like an obvious benefit).

I hacked up a python script to do this on my film collection. It was before mplayer2 so I could only track to keyframes. It was janky but kinda cool to search for a word (say "love") and see a bunch of clips with that word in it. The problem is that subtitles rarely have proper grammar and it's very hard to isolate full phrases.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#32

Post by fbnt »

http://voxforge.com have been collecting a big speech corpora over the last few years, under GPL license. That should be the way to follow imho.

Training a speech recognition engine is quite a sophisticated process, and usually requires at least a clean (not noisy) set of samples, which you can't find in dubbed movies and surely not in music.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#33

Post by tcarnell »

sycren wrote: post #26
nvictor wrote: post #21

Earlier quoted context omitted.

i'm interested.

I would be interested in what you might come up with

The problem with implementing this idea is which technology would actually have to be built, and in which technology the real value would lie. Once the technology is established, scanning videos, tv, radio etc for any source of spoken audio and building up a database of indexable dialogs would almost be the easy part.

Building a speech reconizer is not only difficult, it has also been attempted many times before and unless a speech recognition guru could bring something new to the world, the best we could do is what is already available - so probably best to use existing technology, which often is not cheap to get a license. This is also true with voice print technologies.

The key to getting this up and running lies in finding or building a really good speech recognizer and voice print generator/varifier...

Maybe this is something Y Combinator would be interested in funding? I am based in Europe (Spain at the moment) and I think it would be really hard to convince people to fund this type of technology over here.

If anybody is up for the challenge, I'd love to be involved!

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#34

Post by fraser »

hartror wrote: post #5

Well I am sure they would do, though subtitles aren't the most reliable source for movie dialog. Often the dialog is altered subtly to fit the space and timing requirements.

I've had the subtitles turned on for about a year now and it wouldn't take more than 2 hours of watching broadcast TV with subtitles to realize this isn't a good solution. I've noticed the following.

1. Audio track is censored, Subtitles are not or Vice/Versa. 2. Actors Improvise the audio, the Subtitles are based on the script. 3. English Translations were done by the cheapest person possible so lots of partial words because they weren't clear and the transcriber didn't understand the context. 4. A recent show (2011) seemed to have a symbol every other character, I'm not sure if this is a Double-Byte Character issue, or just a bad translation. 5. Several shows such as American Idol and America's Got Talent display song lyrics and I'm not sure but I would think singing would require changes to the Algorithm.

I wish you well with the idea, but now you have a little more information.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#35

Post by molloye »

tcarnell wrote: post #11

I used to work for a company that built Speech Recognition systems and I came up with a similar/related idea - the idea being to take a load of videos of barack obama (for example), and create an accurate 'voice print'. Once done, any videos or speech could be scanned and if Barack Obama's voice print was recognized/detected, the recognizer could be tuned to his voice print AND could apply a set of appropriate gramma…

Google is doing something comparable with Google Voice and Search recognition transcriptions, inviting corrections both manually and by using similar techniques to spell correct in text search.

I suspect a lack of data is the biggest challenge in improving speech recognition

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#36

Post by molloye »

tcarnell wrote: post #11

I used to work for a company that built Speech Recognition systems and I came up with a similar/related idea - the idea being to take a load of videos of barack obama (for example), and create an accurate 'voice print'. Once done, any videos or speech could be scanned and if Barack Obama's voice print was recognized/detected, the recognizer could be tuned to his voice print AND could apply a set of appropriate gramma…

Google is doing something comparable with Google Voice and Search recognition transcriptions, inviting corrections both manually and by using similar techniques to spell correct in text search.

I suspect a lack of data is not the biggest challenge in improving speech recognition

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#37

Post by killa_bee »

I happen to know that they do this at the Linguistics Data Consortium (http://www.ldc.upenn.edu/), at least with cable news shows. They mostly do that to obtain data for languages with more minimal resources though, and for the purposes of transcription, not for speech recognition qua engineering research. The real issue though is the research community is interested in increasing the accuracy of recognizers on standard datasets by developing better models, not increasing accuracy per se. Having used more data isn't publishable. Further, in terms of real gains, the data is sparse (power law distributed), and so we need more than just a constant increase in the amount of data. This issue is general to any machine-learning scenario but is particularly pronounced in anything built on language.

Some related papers ~ Moore R K. 'There's no data like more data (but when will enough be enough?)', Proc. Inst. of Acoustics Workshop on Innovation in Speech Processing, IoA Proceedings vol.23, pt.3, pp.19-26, Stratford-upon-Avon, 2-3 April (2001). Charles Yang. Who's afraid of George Kingsley Zipf? Ms., University of Pennsylvania. http://www.ling.upenn.edu/~ycharles/papers/zipfnew.pdf

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#40

Post by chc »

sycren wrote: post #20
amirmc wrote: post #19

Earlier quoted context omitted.

Off-topic: I understand your point but I've come to realise that good ideas are not a 'dime-a-dozen'. Mediocre ideas, maybe (although telling them apart is not necessarily obvious).

I would also say that with this particular idea, I have no idea whether you could data mine films and other media without having to pay some hefty licence fees that may perturb startups without enough funding.

I don't believe there's any laws that would mandate this, at least in America. As long as you're merely analyzing legitimately acquired content, I don't see how anybody else has any say in what you're doing. Copyright isn't an absolute control over what can and can't be done with your work — it's just a right to restrict reproduction.
Post reply on HN