AI could end foreign-language subtitles • Hacker News Board

Re: AI could end foreign-language subtitles

#21

Post by Jach » Wed, Jan 19, 2022, 10:38 PM UTC

xiii1408 wrote: post #13

xiii1408 wrote: post #10

Earlier quoted context omitted.

Yes! My perennial problem is that most Chinese TV content has excellent subtitles, but they're burned in to the videos. So if I want to watch a Chinese show with a friend who doesn't read Chinese, there's no option even for auto-translated subtitles. I've often thought of writing a script to generate subtitle files using text recognition, but haven't gotten around to it.

There's also the problem of content which doesn't have subtitles to begin with---YouTube's automatic subtitle generation is great, but could be improved and expanded to languages other than English.

It is expanded, at least to Japanese. What's also fun is you can then have it auto translate the auto generated subs. The results are... not typically great, but the potential!

Re: AI could end foreign-language subtitles

#22

Post by stormbrew » Wed, Jan 19, 2022, 10:39 PM UTC

Swizec wrote: post #6

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

Coming from a non english country I can assure you that dubbing is the worst. When we were kids we would actively avoid watching anything that was dubbed as soon as we could read. It’s actually funny seeing how frustrated with subtitles my American girlfriend gets when we watch anything foreign. “You mean this basic skill every 7 year old has is hard for you, an educated super smart 30 year old? Bwahahaha” I still pr…

I have complicated feelings about this. I also watch most things with subtitles these days, because either (or probably both) my hearing is going a bit or sound design/editing in films and tv shows has gotten worse, but there is a non-trivial difference in how I perceive things with or without.

With subtitles my eyes are drawn more often to a part of the screen where nothing visual is happening. They aren't glued there, but it does create some bias. I have occasionally rewatched a scene without subtitles and noticed a lot of subtlety in the performance I had missed the first time though.

That said, a bad dub is at least as distracting. Probably much more so, if anything. I would rather watch a bad sub than a bad dub, though I have occasionally enjoyed dubbed things as well. I don't think it's an all or nothing thing, but "never watch dubs" is the hard line heuristic that makes sense to me if you're gonna go that way.

Re: AI could end foreign-language subtitles

#23

Post by donkarma » Wed, Jan 19, 2022, 10:39 PM UTC

Yeah I forgot the only use of subtitles is for hearing people

Re: AI could end foreign-language subtitles

#24

Post by openknot » Wed, Jan 19, 2022, 10:39 PM UTC

Swizec wrote: post #6

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

Coming from a non english country I can assure you that dubbing is the worst. When we were kids we would actively avoid watching anything that was dubbed as soon as we could read. It’s actually funny seeing how frustrated with subtitles my American girlfriend gets when we watch anything foreign. “You mean this basic skill every 7 year old has is hard for you, an educated super smart 30 year old? Bwahahaha” I still pr…

I share the same experience of preferring subtitles over dubs, coming from an English country. This is most popular with Japanese media like anime (like movies like 'Your Name'/'Kimi no wa na' or Attack on Titan). It also fits for foreign language films (e.g. preferring the original German for The Lives of Others/Wings of Desire).

Crucially, outside of preference, subtitles are occasionally necessary for translating text on the screen (e.g. on building signs, newspaper front pages, posters, chalkboard writing), as information is conveyed solely through writing without spoken dialogue.

For example, consider the chalkboard scene in "Komi san can't communicate": https://www.youtube.com/watch?v=JoH8I-NC5tM . It's key to the story that a character has trouble communicating verbally, so subtitles are essential to understand the written communication on the chalkboard. (You can then compare to a more extensively subtitled version that translates all the text on the board; good subtitles make a difference to storytelling: https://old.reddit.com/r/anime/comments/qdjgba/the_blackboar...).

Re: AI could end foreign-language subtitles

#25

Post by kingkawn » Wed, Jan 19, 2022, 10:39 PM UTC

I want to watch the original language with text, not listen to english adaptation whether synthesized or acted.

Re: AI could end foreign-language subtitles

#26

Post by sp332 » Wed, Jan 19, 2022, 10:40 PM UTC

endisneigh wrote: post #12

What’s really surprising to me is that synching subtitles isn’t a solved problem. Why can’t someone just loosely transcribe without time stamps and sync it to the video?

I think you can do this with YouTube.

Re: AI could end foreign-language subtitles

#27

Post by pessimizer » Wed, Jan 19, 2022, 10:41 PM UTC

Swizec wrote: post #6

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

Coming from a non english country I can assure you that dubbing is the worst. When we were kids we would actively avoid watching anything that was dubbed as soon as we could read. It’s actually funny seeing how frustrated with subtitles my American girlfriend gets when we watch anything foreign. “You mean this basic skill every 7 year old has is hard for you, an educated super smart 30 year old? Bwahahaha” I still pr…

As an english first language speaker, I even find english subtitles on english language media very distracting. I consume a huge amount of subtitled films/tv because most good things aren't in english, but subtitles keep my eyes focused on the bottom of the screen.

I don't like dubbing because it's generally much lower quality than the original performance, but I get why people who don't have a problem reading could prefer it.

Re: AI could end foreign-language subtitles

#28

Post by cfcosta » Wed, Jan 19, 2022, 10:41 PM UTC

ahmed_ds wrote: post #15

On a tangential note: As far as training data for text and speech driven machine learning models go, isn't srt files truly remarkable? They are time stamped hence annotated precisely, hash and file matching makes media content to subtitle matching exact and overall there are language variations.

They are very artisanal, though. I used to do timing for anime fansubs when I was a teenager, and there's a lot of subtlety going on, specially when you don't speak the language (you're guided mostly by sound and phonetics, as the translations come more or less untimed).

Timing is much more automatized nowadays, but still doesn't feel as high quality as handicrafted timing does.

Re: AI could end foreign-language subtitles

#29

Post by moffkalast » Wed, Jan 19, 2022, 10:42 PM UTC

Bilal_io wrote: post #19

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

I can confirm as an anime fan, when watching One Piece I prefer to watch it in Japanese as the dubbed version sounds odd. it was an advantage growing up watching Hollywood movies in English with Arabic subtitles, it helped me learn the language.

It also helps when what you're watching has atrocious audio mixing (looking at you Nolan), or you have shit speakers, or actors mumble or whisper, in which case it's helpful to have them as an aid when something's not clear.

Re: AI could end foreign-language subtitles

#30

Post by kashunstva » Wed, Jan 19, 2022, 10:42 PM UTC

I feel the same way about this as I do about reading poetry in translation, or listening to the (fortunately rare) opera sung in translation. There's something integral about the native qualities of the original language that I can't imagine would survive the AI process.