Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#1

Post by sycren » Tue, Aug 30, 2011, 9:14 AM UTC

There are thousands of films and tv episodes that have subtitles throughout their duration. Millions of music that are sung that we can find lyrics for. Would it not be possible to use this material to train speech recognition. This would then make it possible to train in the multiple different dialects and accents of a particular language.

Speech recognition as a technology, has always appeared to move slowly although with the advent of mobile popularity, the technology is becoming increasingly popular.

Is anyone doing anything like this?

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#2

Post by drKarl » Tue, Aug 30, 2011, 9:21 AM UTC

There are two different but correlated fields: Speech recognition and Natural Language Understanding.Speech Recognition is easier if the scope is minimised, that is, if th system knows which subset of keywords of orders to recognized. But recognizing an open scope, including different accents, slang, etc is a much more difficult task.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#3

Post by SandB0x » Tue, Aug 30, 2011, 9:34 AM UTC

A similar idea: "Learning sign language by watching TV (using weakly aligned subtitles" from CVPR 2009:

http://www.comp.leeds.ac.uk/me/Publications/cvpr09_bsl.pdf

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#4

Post by sycren » Tue, Aug 30, 2011, 9:43 AM UTC

drKarl wrote: post #2

There are two different but correlated fields: Speech recognition and Natural Language Understanding.Speech Recognition is easier if the scope is minimised, that is, if th system knows which subset of keywords of orders to recognized. But recognizing an open scope, including different accents, slang, etc is a much more difficult task.

I mean there must be millions of times where a character has said 'hello' in a film or tv episode. Each person may have a slightly different way of saying it which can then be used to make a model for speech recognition software which may no longer require the user to train the software.

It may also be possible to automate the entire process as we have both the audio and the words spoken at a particular time.

Take it a step further, we have millions of sung songs with lyrics that can also be used. Its a gold mine of information that can be repurposed.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#5

Post by hartror » Tue, Aug 30, 2011, 9:44 AM UTC

Well I am sure they would do, though subtitles aren't the most reliable source for movie dialog. Often the dialog is altered subtly to fit the space and timing requirements.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#6

Post by sycren » Tue, Aug 30, 2011, 9:50 AM UTC

hartror wrote: post #5

Well I am sure they would do, though subtitles aren't the most reliable source for movie dialog. Often the dialog is altered subtly to fit the space and timing requirements.

How about music lyrics?

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#7

Post by mooism2 » Tue, Aug 30, 2011, 9:52 AM UTC

For the purposes of speech recognition, songs strike me as being particularly noisy.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#8

Post by hartror » Tue, Aug 30, 2011, 9:58 AM UTC

sycren wrote: post #6

hartror wrote: post #5

Well I am sure they would do, though subtitles aren't the most reliable source for movie dialog. Often the dialog is altered subtly to fit the space and timing requirements.

How about music lyrics?

For training speech recognition? Songs strikes me as poor training data.

I wasn't arguing against movies, just that subtitles rather than a final script isn't the best data source.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#9

Post by uniclaude » Tue, Aug 30, 2011, 10:06 AM UTC

mooism2 wrote: post #7

For the purposes of speech recognition, songs strike me as being particularly noisy .

True, we may think about acappellas then.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#10

Post by tcarnell » Tue, Aug 30, 2011, 10:18 AM UTC

Very interesting. In a similar vein, automated language translation could be assisted too - beacuase a film DVD often has different audio and subtitle languages, so it would be possible to pair up semantically similar audio and written content... and put it all into a magic computer