Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-source tool translates and dubs videos into other languages using AI

github.com

Post reply on HN
Print view View original

41–50 of 119 posts

Re: Open-source tool translates and dubs videos into other languages using AI

#41

Post by ZoomerCretin »

pizza234 wrote: post #36
Kiro wrote: post #13

Earlier quoted context omitted.

Strange submission to post this comment on. It's not like translating and dubbing videos is the highest form of labor.

Voice acting (dubbing) is acting, which is a "high form of labor". In some languages/countries, most of the dubbing is performed by a few extremely good voice actors.

Watch Squidgame's English dubs and tell me that couldn't be improved.

If for no other reason, this tool is a net-positive because it may raise the bar on the quality of dubs expected from dubbing actors.

Re: Open-source tool translates and dubs videos into other languages using AI

#42

Post by joe_the_user »

pizza234 wrote: post #36
Kiro wrote: post #13

Earlier quoted context omitted.

Strange submission to post this comment on. It's not like translating and dubbing videos is the highest form of labor.

Voice acting (dubbing) is acting, which is a "high form of labor". In some languages/countries, most of the dubbing is performed by a few extremely good voice actors.

LLMs aren't going replace all actors or all voice actors just as they won't replace all illustrators or writers.

LLMs provide a certain level of mid/low quality content in nearly all mediums. And given that there many people producing such mid/low quality content today, LLMs will have an impact. LLMs affecting sales writers? Sure (not the best sales writers but the point is sales is mediocre but acceptable is a norm).

And say LLMs specifically. There's good evidence the technology has roughly peaked. That doesn't mean it's impact has peaked but it's indication that "all jobs at risk" might be an exaggeration.

Re: Open-source tool translates and dubs videos into other languages using AI

#43

Post by sunnybeetroot »

cubbic wrote: post #4

Oh I made something similar but for Netflix! https://github.com/cubbK/dubbing_ai_netflix_client I want to learn swedish and because there are so few dubbed movies in Swedish I take the subtitles(Netflix is good at having subtitles in different languages) and text-to-speech it :)

This is fantastic! Any direction on if I wanted to change the language to something else?

Re: Open-source tool translates and dubs videos into other languages using AI

#45

Post by mrtksn »

Back in high school, when I got my first PC a plumber came over to fix some stuff and when he saw the computer he got excited and asked some questions and one of the questions was “how do you translate the VCD with this, I have a movie to watch but hate subtitles”.

I was like “silly dude doesn’t know how computers work” but maybe I was the silly one who can’t dare to imagine how something like that can work.

Re: Open-source tool translates and dubs videos into other languages using AI

#46

Post by codedokode »

As I understand, it first extracts text from original video into subtitles, translates them using external LLM, and then converts text to speech. All of this is done using thrid-party solutions, and the project seems to be just a GUI app that allows to integrate them.

You obviously cannot use this to translate songs or movies because this method loses important information like voice, intonation, etc.

So it is still better to use subtitles.

Re: Open-source tool translates and dubs videos into other languages using AI

#47

Post by codedokode »

Kiro wrote: post #13
vouaobrasil wrote: post #2

Unlike other methods of automation, AI is replacing human beings too fast. And before you say, "new jobs will be created" -- look at history. After the computer, new jobs have been created, but what kind of jobs? Every year, we are becoming more entwined in wage slavery as the wealth accumulates at the top and jobs become more meaningless. So, no, new jobs will not be created, except the kind of jobs that crush the h…

Strange submission to post this comment on. It's not like translating and dubbing videos is the highest form of labor.

It is a form of art. I hate watching dubbed videos, because the voice actors who translate it almost always are worse than original actors.

Re: Open-source tool translates and dubs videos into other languages using AI

#48

Post by amonith »

IndySun wrote: post #32
vouaobrasil wrote: post #2

Unlike other methods of automation, AI is replacing human beings too fast. And before you say, "new jobs will be created" -- look at history. After the computer, new jobs have been created, but what kind of jobs? Every year, we are becoming more entwined in wage slavery as the wealth accumulates at the top and jobs become more meaningless. So, no, new jobs will not be created, except the kind of jobs that crush the h…

>...new jobs will not be created, except the kind of jobs that crush the human spirit into oblivion... AI certainly means everyone will be able to create 'art' and as a result we'll have more art than we know what to do with, music and images are already confetti, soon so will full length 'films/movies'. That leaves anyone who can actually sing, paint, play, dance, in prime position to take up those mantles.

What do you mean by the last sentence? Isn't it the opposite? People who could do art stuff won't be able to sustain themselves using those skills anymore because those will be devalued a lot.

Re: Open-source tool translates and dubs videos into other languages using AI

#49

Post by woodson »

programjames wrote: post #14
netsharc wrote: post #12

Huh, if it's good enough, movies/TV shows dubbed with AI-clone of the original voice would be great (if we can ignore the ethics of using the actor's voice and the loss of work for the dubbing companies and actors). For example here's how weird Friends is in German: https://www.youtube.com/watch?v=nCoNSZV--z0 . Or Italian: https://www.youtube.com/watch?v=wO5qTzvyQ1s Can AI detect the emotional tone of sentences yet,…

Yes, pop in a normalizing flow that removes tone and recreates it with a small audio sample as context. https://arxiv.org/abs/2312.01479

What I found is that, for cross-language use-cases, this often just applies the intonation of the “context” sample to the created sample, which, if they are from different languages, usually gives the wrong result (in the sense that it sounds off).

Re: Open-source tool translates and dubs videos into other languages using AI

#50

Post by ranger_danger »

microflash wrote: post #10
ranger_danger wrote: post #6

An English README would have been nice

Probably you missed it: https://github.com/jianchang512/pyvideotrans/blob/main/docs/...

Yep I missed the "English" text right on the first line, sorry.
Post reply on HN