Live data from Hacker News

Ask HN: Why don't we use subtitled films/tv to train speech recognition?

news.ycombinator.com

21–30 of 40 posts

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#21

I used to work for a company that built Speech Recognition systems and I came up with a similar/related idea - the idea being to take a load of videos of barack obama (for example), and create an accurate 'voice print'. Once done, any videos or speech could be scanned and if Barack Obama's voice print was recognized/detected, the recognizer could be tuned to his voice print AND could apply a set of appropriate gramma…

i'm interested.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#22
post #16

I used to work for a company that built Speech Recognition systems and I came up with a similar/related idea - the idea being to take a load of videos of barack obama (for example), and create an accurate 'voice print'. Once done, any videos or speech could be scanned and if Barack Obama's voice print was recognized/detected, the recognizer could be tuned to his voice print AND could apply a set of appropriate gramma…

> when you google for text, you could actually retrieve videos whose content exactly matches the search terms and jump directly to that part of the video The search aspect of this is very interesting and I hadn't thought of it before (though in hindsight it seems like an obvious benefit).

[deleted]

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#23

Aside from issues relating to background noise on the soundtrack, the subtitles are frequently abridged from the spoken word in the interests of space and / or readability, so you'd need to account for that in your algorithm. If it were me... Project Gutenberg has free books available in both audio and text formats. You may well again run into issues with the spoken and written text not exactly matching (it's not som…

Audio recordings of book readings are less practical than subtitles because they are not synchronized. Every subtitle in a film is associated with the sound clip that plays while it is visible, whereas for an audiobook or similar, any algorithm would have to "align" the audio and text in order to obtain usable training data, and then it would have to deal with the errors introduced by this process.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#24

Aside from issues relating to background noise on the soundtrack, the subtitles are frequently abridged from the spoken word in the interests of space and / or readability, so you'd need to account for that in your algorithm. If it were me... Project Gutenberg has free books available in both audio and text formats. You may well again run into issues with the spoken and written text not exactly matching (it's not som…

Audio recordings of book readings are less practical than subtitles because they are not synchronized. Every subtitle in a film is associated with the sound clip that plays while it is visible, whereas for an audiobook or similar, any algorithm would have to "align" the audio and text in order to obtain usable training data, and then it would have to deal with the errors introduced by this process.

Hmm, good point, though still against a cleaner audio signal, easier input data to process and (likely) closer matching of text. I'm not remotely involved with the field so I won't indulge in further wild speculation, but an interesting balance.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#25
For films and music the audio data may have too much noise, but TV programmes with low background noise (news, documentary, interview) with available Closed Captions (CC) are good training sources. CC transcripts are enforced by broadcasting regulators so they should be highly accurate.

The big problem with using these sources is the huge vocabulary. Speech recognition works better for smaller vocabularies than bigger.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#26
post #21

I used to work for a company that built Speech Recognition systems and I came up with a similar/related idea - the idea being to take a load of videos of barack obama (for example), and create an accurate 'voice print'. Once done, any videos or speech could be scanned and if Barack Obama's voice print was recognized/detected, the recognizer could be tuned to his voice print AND could apply a set of appropriate gramma…

i'm interested.

I would be interested in what you might come up with

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#27

Aside from issues relating to background noise on the soundtrack, the subtitles are frequently abridged from the spoken word in the interests of space and / or readability, so you'd need to account for that in your algorithm. If it were me... Project Gutenberg has free books available in both audio and text formats. You may well again run into issues with the spoken and written text not exactly matching (it's not som…

As 0x12 states further down, noise can be seen as beneficial. By having such a huge dataset, perhaps it would be possible to advance the technology of speech recognition to transcribe speech in busy places as needed for in mobile applications where the user is not in a quiet room.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#28
post #27

Aside from issues relating to background noise on the soundtrack, the subtitles are frequently abridged from the spoken word in the interests of space and / or readability, so you'd need to account for that in your algorithm. If it were me... Project Gutenberg has free books available in both audio and text formats. You may well again run into issues with the spoken and written text not exactly matching (it's not som…

As 0x12 states further down, noise can be seen as beneficial. By having such a huge dataset, perhaps it would be possible to advance the technology of speech recognition to transcribe speech in busy places as needed for in mobile applications where the user is not in a quiet room.

Perhaps, but I wouldn't use that as a starting dataset; noise resilience and a training set for enhancing this functionality is surely better developed on top of a working implementation for a lower-noise input? Better to build the easier solution and reinforce it for hard problems than try to go straight at the hard problems.

Put it another way; which would you start by teaching a student: the easy situations or the more complex situations?

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#29
post #14
post #5

Well I am sure they would do, though subtitles aren't the most reliable source for movie dialog. Often the dialog is altered subtly to fit the space and timing requirements.

And they are translations not speech-2-text.

Movies are also subtitled in the same language for the hearing impaired.

Re: Ask HN: Why don't we use subtitled films/tv to train speech recognition?

#30
post #16

I used to work for a company that built Speech Recognition systems and I came up with a similar/related idea - the idea being to take a load of videos of barack obama (for example), and create an accurate 'voice print'. Once done, any videos or speech could be scanned and if Barack Obama's voice print was recognized/detected, the recognizer could be tuned to his voice print AND could apply a set of appropriate gramma…

> when you google for text, you could actually retrieve videos whose content exactly matches the search terms and jump directly to that part of the video The search aspect of this is very interesting and I hadn't thought of it before (though in hindsight it seems like an obvious benefit).

[deleted]
Post reply on HN