Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI could end foreign-language subtitles

axios.com

Post reply on HN
Print view View original

141–150 of 221 posts

Re: AI could end foreign-language subtitles

#141

Post by nsxwolf »

tompetry wrote: post #80
aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

I am very surprised more people watched the dubbed version of Squid Game over the subtitles. I saw a friend watching the dubbed version and to me it didn't "preserve the original performances" as you say. Surprised subtitles are a barrier, I enjoy them personally, but I guess they have the data to know. I have tested out services that translate your "own voice" to another language and it was pretty cool, although I c…

I love the dubbed version. The performances are exaggerated and full of character. The have the feel like a live action (dubbed) anime.

Re: AI could end foreign-language subtitles

#144

Post by esjeon »

I’m highly skeptical about AI-replacing-subtitle-as-a-whole. We all know what’s the problem: the quality. We already know that translation is difficult. We already know that acting is difficult.

The only feasible outcome here is deepfaking the voice from the recordings from localized voice actors to the actual movie actors. Perhaps, in the future, a voice actor might be doing all of (1) translation (2) voice acting (3) deepfaking in one shot. One person might be able to dub the whole movie - alone.

But, for sure, we won’t be able to remove human from the process.

Re: AI could end foreign-language subtitles

#145

Post by derefr »

derekp7 wrote: post #132
creakingstairs wrote: post #89

Earlier quoted context omitted.

When I've moved to New Zealand, I was taken aback at how many people just could not stand subtitles. This and your mention about 'barrier' reminded me of Bong Joon Ho (director of Parasite) calling this out: "Once you overcome the 1-inch-tall barrier of subtitles, you will be introduced to so many more amazing films."

The problem I have with subtitles is the same reason I could never enjoy comics / graphic novels. I can either read and process the text, or look at / process the action (pictures). For me it is the same thing as trying to pay attention to two conference calls at the same time with a phone in each ear (is it just me that can't do this?). And it isn't just the mental processing, the rest of the screen sort of disappea…

I think this is why people mostly talk about watching foreign-language animated works and soap operas/dramas: dramas will have either action or talking heads, but not usually both; and animation can be paced stylistically, so that each shot lasts long enough "by design" to read the subtitles, even when it would be over in a split second in reality.

But I can't imagine watching, say, a foreign live-action sketch comedy show subtitled. I'd have exactly the bad experience you're describing.

Re: AI could end foreign-language subtitles

#146

Post by Gigachad »

tompetry wrote: post #80
aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

I am very surprised more people watched the dubbed version of Squid Game over the subtitles. I saw a friend watching the dubbed version and to me it didn't "preserve the original performances" as you say. Surprised subtitles are a barrier, I enjoy them personally, but I guess they have the data to know. I have tested out services that translate your "own voice" to another language and it was pretty cool, although I c…

For me, reading the subtitles takes away from the experience much more than any kind of loss in authenticity does. Keeping my eyes on the text takes away from the visual.

Re: AI could end foreign-language subtitles

#147

Post by mcoliver »

Couple thoughts:

- The hard part about subtitles is not translation it's contextualization. Languages are not variable replacements and words have different meanings based on the context of the sentence. Languages have nuance, phrases, and slang unique to their cultures which further adds difficulty.

- coupled with the above, permutations can get large with translated dubs. Which subtitle are you supposed to show? Most translated subs are done off the original audio track and the dubs are done separately. So for example with an English show, the English subs match the English track, but the translation will be off for the Spanish audio track. And the Spanish dubs will not match the Spanish subs either.

- lip sync on dubs is a hard problem for many reasons ( full face and body emotion, how words are said in the native tounge, etc..) and will significantly improve the viewing experience. You can do creative things like have fewer full face shots to help with this. It's also important to get the audio mix right so it doesn't sound like an extra track. Keep in mind you now need more videos to cache and store and the ability to swap video streams when you swap audio streams.

- animated content offers other approaches than leveraging nn/deepfakes because the assets are already rigged. The use of game engines offers some really interesting opportunities here.

Re: AI could end foreign-language subtitles

#148

Post by derefr »

commandlinefan wrote: post #39
aeturnum wrote: post #3

This would be a lovely option to have and I find it hilarious that anyone imagines that subtitles are some kind of problem we are trying to get out of. Adding text to movies has a huge array of advantages: it preserves the original performances, it makes the dialog accessible to people with hearing or auditory processing disabilities, they can be multiplexed in a way that audio cannot, etc. Obviously it would be wond…

> subtitles are some kind of problem Heck, they're not even limited to foreign languages. I put them on in English because I'm starting to lose my hearing now in my late 40's and if I turn up the volume loud enough that I can hear the dialogue ok, the sound effects would be so loud they'd set off the neighbor's car alarm.

If you can afford it, buy a speaker system with at least a dedicated center channel. Dialog is almost always mixed to the center channel in 5.1/7.1 mixes. Which means that you can just turn up the gain on the center channel speaker, while leaving everything else down. The result is magical — it's like turning down the "background music" slider in a video game :)

Re: AI could end foreign-language subtitles

#149

Post by Gigachad »

throw_away wrote: post #5

I would much rather we use AI to improve subtitling. Better auto speech recognition, per-word millisecond synchronization, positioning based on speaker/screen content-- all these could improve the subtitled experience and would be way less creepy than deepfake voices.

Both are happening. Google meet has live auto subtitles that are pretty accurate. Sometimes I find even as an English speaker I am able to understand through the compression better with the subtitles on.

Re: AI could end foreign-language subtitles

#150

Post by Gigachad »

bovermyer wrote: post #133

A couple points here. 1. AI might be able to prevent bad dubbing/subbing, but it will not be able to compete with _good_ dubbing/subbing in our lifetimes. There's an artistry there that no AI can match. 2. I love subtitles and prefer original audio + English subtitles in anything foreign-language. Besides the voices matching up to the movement of the actors' lips and thus not breaking my immersion, I also adore heari…

> There's an artistry there that no AI can match.

This has been said a few times already and been proven false. The quality of GANs is moving at a staggering pace.

Post reply on HN