AI could end foreign-language subtitles • Hacker News Board

Re: AI could end foreign-language subtitles

#81

Post by robocat » Thu, Jan 20, 2022, 1:27 AM UTC

tlbsofware wrote: post #72

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

Personally I would like if someone could use AI to enhance the voices of the actors, I find a lot of movies like to use loud ominous music while the characters are casually talking in a soft voice. Really drives me insane, and placing subtitles helps this problem but it creates a new one, I tend to get distracted by the subtitles and find myself not watching the movie

a way to remove the mood music and leave the voices and foley.

I really enjoyed the cinematography of a film the other day, but the music completely ruined it.

Re: AI could end foreign-language subtitles

#82

Post by pessimizer » Thu, Jan 20, 2022, 1:28 AM UTC

mateo1 wrote: post #59

endisneigh wrote: post #12

What’s really surprising to me is that synching subtitles isn’t a solved problem. Why can’t someone just loosely transcribe without time stamps and sync it to the video?

What's the market for this? Very small.

Literally everyone who subtitles anything.

Re: AI could end foreign-language subtitles

#83

Post by pessimizer » Thu, Jan 20, 2022, 1:33 AM UTC

testermelon wrote: post #35

The notion that some people prefer dub over sub is not new, but it baffles me that it is to the level of urgent business concern. From what I see in other comments, is it an American thing? We should invest in making people’s ear accustomed/used to hear other languages. It has great intercultural benefits.

I'm pretty sure it's also a Russian thing.

In the case of English, the orthography is difficult, a large proportion of people are functionally illiterate, and even more have a tough enough time reading that subtitles just go by too quickly.

Re: AI could end foreign-language subtitles

#84

Post by Bedon292 » Thu, Jan 20, 2022, 1:35 AM UTC

jonpurdy wrote: post #56

nkozyra wrote: post #42

Earlier quoted context omitted.

> I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. I think there's a lot of gatekeeping about this. People - perhaps rightfully so - believe subtitled versions to be more ... "pure" than dubbed versions. I don't watch any language dubbed movies, but I wouldn't chide anyone who did. Not everyone wants to spend the whole two hours glancing up and down across…

> I think there's a lot of gatekeeping about this. People - perhaps rightfully so - believe subtitled versions to be more ... "pure" than dubbed versions. Well, they are more pure than dubbed versions. There is so much nuance and tone in the original language that is either left out or changed in the dub. Not to mention word order (subject-verb-object in English/Chinese, subject-object-verb in Japanese/Korean) that f…

When I was a kid, my dad was stationed in Japan. We watched all kinds of stuff, including Totoro on Japanese TV. No subtitles or dubs. Not a clue what they were actually saying but we all enjoyed it anyways. So you could always try it out and see how it goes.

Re: AI could end foreign-language subtitles

#85

Post by earthboundkid » Thu, Jan 20, 2022, 1:36 AM UTC

jonpurdy wrote: post #56

nkozyra wrote: post #42

Earlier quoted context omitted.

> I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. I think there's a lot of gatekeeping about this. People - perhaps rightfully so - believe subtitled versions to be more ... "pure" than dubbed versions. I don't watch any language dubbed movies, but I wouldn't chide anyone who did. Not everyone wants to spend the whole two hours glancing up and down across…

> I think there's a lot of gatekeeping about this. People - perhaps rightfully so - believe subtitled versions to be more ... "pure" than dubbed versions. Well, they are more pure than dubbed versions. There is so much nuance and tone in the original language that is either left out or changed in the dub. Not to mention word order (subject-verb-object in English/Chinese, subject-object-verb in Japanese/Korean) that f…

I have been watching dubs of Ghibli with my kids. It’s fine. It would be better to watch the subs for me, but I’m not going to get bent out of shape about it. They usually have pretty good English voice casting. I kind of like Kirstin Dunst as Kiki.

Re: AI could end foreign-language subtitles

#86

Post by suthakamal » Thu, Jan 20, 2022, 1:36 AM UTC

An idea only a single-language speaker could love.

Re: AI could end foreign-language subtitles

#87

Post by Delk » Thu, Jan 20, 2022, 1:36 AM UTC

jonpurdy wrote: post #56

nkozyra wrote: post #42

Earlier quoted context omitted.

> I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. I think there's a lot of gatekeeping about this. People - perhaps rightfully so - believe subtitled versions to be more ... "pure" than dubbed versions. I don't watch any language dubbed movies, but I wouldn't chide anyone who did. Not everyone wants to spend the whole two hours glancing up and down across…

> I think there's a lot of gatekeeping about this. People - perhaps rightfully so - believe subtitled versions to be more ... "pure" than dubbed versions. Well, they are more pure than dubbed versions. There is so much nuance and tone in the original language that is either left out or changed in the dub. Not to mention word order (subject-verb-object in English/Chinese, subject-object-verb in Japanese/Korean) that f…

There's often a lot of nuance lost in translated subtitles as well, in my experience, often because dialogue written for one language can't be coerced into natural dialogue in a different language without significant changes.

Keeping the original audio track does preserve the direct relationship between the actor's tone and expressions, of course, and it could also be something closer to what the original makers intended even if the voice actors in a dub also do a good job.

The audio track also feels somehow more right and authentic when someone who's supposedly Japanese speaks Japanese, or when a supposedly French person speaks French, or when people in a show set in California speak English rather than somehow magically speaking your native language.

Then there's of course the question of dealing with scripts that have multiple languages spoken in them. Something would surely be lost if multiple languages were dubbed into the same third language. It's often easy to recognize the different languages being spoken even if one does not understand them.

Re: AI could end foreign-language subtitles

#88

Post by BlueTemplar » Thu, Jan 20, 2022, 1:40 AM UTC

xiii1408 wrote: post #13

xiii1408 wrote: post #10

Earlier quoted context omitted.

Yes! My perennial problem is that most Chinese TV content has excellent subtitles, but they're burned in to the videos. So if I want to watch a Chinese show with a friend who doesn't read Chinese, there's no option even for auto-translated subtitles. I've often thought of writing a script to generate subtitle files using text recognition, but haven't gotten around to it.

There's also the problem of content which doesn't have subtitles to begin with---YouTube's automatic subtitle generation is great, but could be improved and expanded to languages other than English.

Sadly, they seem instead to have been recently removing this functionality (as well as auto-translation of those).

Re: AI could end foreign-language subtitles

#89

Post by creakingstairs » Thu, Jan 20, 2022, 1:41 AM UTC

tompetry wrote: post #80

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

I am very surprised more people watched the dubbed version of Squid Game over the subtitles. I saw a friend watching the dubbed version and to me it didn't "preserve the original performances" as you say. Surprised subtitles are a barrier, I enjoy them personally, but I guess they have the data to know. I have tested out services that translate your "own voice" to another language and it was pretty cool, although I c…

When I've moved to New Zealand, I was taken aback at how many people just could not stand subtitles. This and your mention about 'barrier' reminded me of Bong Joon Ho (director of Parasite) calling this out: "Once you overcome the 1-inch-tall barrier of subtitles, you will be introduced to so many more amazing films."

Re: AI could end foreign-language subtitles

#90

Post by zeroimpl » Thu, Jan 20, 2022, 1:42 AM UTC

aeturnum wrote: post #58

jmckib wrote: post #43

Earlier quoted context omitted.

I love subtitles, but for me they tend to spoil the timing of the lines being delivered, especially with comedy. Unfortunately subtitles are necessary even when I'm able to play audio as loud as I want, because it's almost always hard to understand at least 10-20% of the words that actors are saying. Maybe it would be nice to have subtitles that appear a word at a time as they are spoken, but I imagine that's not ver…

I would also love to a more responsive subtitle format! I've seen some really wonderful custom jobs where people use different subtitle colors for different speakers, place subtitles near the speaker in the frame, etc. I think there is a ton of room for improving the expressiveness of the medium.

A pet peeve of mine has been subtitles which flip the order of words, particularly when the sentence is broken into two lines. Like the spoken text in foreign language is “A murderer is... Joe” but it gets translated to “Joe is a... murderer”.

The other day I happened to see live content with closed captions, and I thought I would love the option to see subtitles like this, where words are translated precisely when spoken.