AI could end foreign-language subtitles • Hacker News Board

Re: AI could end foreign-language subtitles

#211

Post by cyphar » Fri, Jan 21, 2022, 2:29 AM UTC

endisneigh wrote: post #12

What’s really surprising to me is that synching subtitles isn’t a solved problem. Why can’t someone just loosely transcribe without time stamps and sync it to the video?

With tools like alass[1] (using it to synchronise against the original language subtitles) it is about as close to solved you can get.

All of the attempts I've seen of using audio information to synchronise subtitles have been awful. One issue is that some languages subtitle everything, even screams and incoherent shouts (such as Japanese) while others only subtitle dialogue and often rework dialogue for the purposes of making the subtitles short enough to be readable easily. It feels like you need too much domain knowledge to know how different languages subtitle things and that subtitles that match the general meaning of what is being said should be matched up.

[1]: https://github.com/kaegi/alass

Re: AI could end foreign-language subtitles

#212

Post by ramphastidae » Fri, Jan 21, 2022, 4:02 AM UTC

derekp7 wrote: post #132

creakingstairs wrote: post #89

Earlier quoted context omitted.

When I've moved to New Zealand, I was taken aback at how many people just could not stand subtitles. This and your mention about 'barrier' reminded me of Bong Joon Ho (director of Parasite) calling this out: "Once you overcome the 1-inch-tall barrier of subtitles, you will be introduced to so many more amazing films."

The problem I have with subtitles is the same reason I could never enjoy comics / graphic novels. I can either read and process the text, or look at / process the action (pictures). For me it is the same thing as trying to pay attention to two conference calls at the same time with a phone in each ear (is it just me that can't do this?). And it isn't just the mental processing, the rest of the screen sort of disappea…

I’m the same. Reading subtitles takes my attention almost entirely away from the scene, and after switching back and forth between subtitles and the scenes rapidly a few times, all sense of immersion is lost. It’s a distraction and chore.

Re: AI could end foreign-language subtitles

#213

Post by Dracophoenix » Fri, Jan 21, 2022, 5:39 AM UTC

Bilal_io wrote: post #19

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

I can confirm as an anime fan, when watching One Piece I prefer to watch it in Japanese as the dubbed version sounds odd. it was an advantage growing up watching Hollywood movies in English with Arabic subtitles, it helped me learn the language.

I guess you were a fan of MBC 2 then?

Re: AI could end foreign-language subtitles

#214

Post by Bilal_io » Fri, Jan 21, 2022, 5:41 AM UTC

Dracophoenix wrote: post #213

Bilal_io wrote: post #19

Earlier quoted context omitted.

I can confirm as an anime fan, when watching One Piece I prefer to watch it in Japanese as the dubbed version sounds odd. it was an advantage growing up watching Hollywood movies in English with Arabic subtitles, it helped me learn the language.

I guess you were a fan of MBC 2 then?

And MBC Action

Re: AI could end foreign-language subtitles

#215

Post by Dracophoenix » Fri, Jan 21, 2022, 5:41 AM UTC

FredPret wrote: post #73

aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

It would be hilarious to see famous actors deepfaked into speaking foreign languages in a feature film. I suspect it simply won't work - translating accurately is really hard, but translating elegantly is frequently impossible. Some things are just easier/punchier/funnier to say in language A than B

Already the case with Tom Cruise and Jack Nicholson speaking in German.

https://www.dailymail.co.uk/sciencetech/article-9613749/AI-c...

Re: AI could end foreign-language subtitles

#216

Post by Dracophoenix » Fri, Jan 21, 2022, 5:47 AM UTC

Bilal_io wrote: post #214

Dracophoenix wrote: post #213

Earlier quoted context omitted.

I guess you were a fan of MBC 2 then?

And MBC Action

Were you a fan of MBC 3?

Re: AI could end foreign-language subtitles

#217

Post by Bilal_io » Fri, Jan 21, 2022, 6:12 AM UTC

Dracophoenix wrote: post #216

Bilal_io wrote: post #214

Earlier quoted context omitted.

And MBC Action

Were you a fan of MBC 3?

Ages ago. But the ads on MBC network... The meme was that you're watching ads and they hit you with a show.

Re: AI could end foreign-language subtitles

#218

Post by Dracophoenix » Fri, Jan 21, 2022, 6:17 AM UTC

Bilal_io wrote: post #217

Dracophoenix wrote: post #216

Earlier quoted context omitted.

Were you a fan of MBC 3?

Ages ago. But the ads on MBC network... The meme was that you're watching ads and they hit you with a show.

Do you remember the name of a cartoon short involving a rotund orange cat walking around the city doing engaging in various hijinks before it arrived home unnoticed? It's been bugging me for ages. It played between shows in place of a commercials in-between shows circa 2006.

Re: AI could end foreign-language subtitles

#219

Post by shkkmo » Sat, Jan 22, 2022, 3:48 PM UTC

zeroimpl wrote: post #197

lisper wrote: post #153

Earlier quoted context omitted.

And the GP's example is the perfect illustration. "A murderer is Joe" sounds really weird. No native English speaker would ever say that. They would say, "Joe is a murderer" or " The murderer is Joe" or even "One of the murderers is Joe." But never "A murderer is Joe." Maybe in a Victorian novel, but not in a movie.

There are certainly tradeoffs, which is why I said I’d like the option. Since I’m moderately familiar with the other language, I really just need the translation for words I don’t understand or mis-hear. There is such a thing as a real-time translation. I’ve heard it in news broadcasts when a politician speaks a foreign language. It also happens all the time at the UN and other international negotiations. It works we…

I don't think real time translation matches word for word either.

Something that might be sort of like what you are looking for is Language Learning with Netflix https://chrome.google.com/webstore/detail/language-learning-...

It's a browser plugin that provides advanced subtitle functionality. You can have dusl sets of subtitles up, or just the original language subtitles with the option to pause and mouse over individual words for the definition.

Re: AI could end foreign-language subtitles

#220

Post by thfuran » Sat, Jan 22, 2022, 4:29 PM UTC

acchow wrote: post #208

thfuran wrote: post #123

Earlier quoted context omitted.

And Netflix in particular seems to love defaulting to 5.1 tracks when it shouldn't.

I don't even see an option to manually select a stereo track anymore. They seem to have removed it from my Smart TV's app.

I don't think I've yet encountered something with an English 5.1 track that doesn't also have an English stereo track mixed in with whatever foreign language tracks are present but who knows what features any given app has.