Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Automatically synchronize subtitles with video

github.com

Post reply on HN
Print view View original

71–80 of 129 posts

Re: Show HN: Automatically synchronize subtitles with video

#71

Post by kakarot »

kumarharsh wrote: post #59

Completely tangential question to this awesome discussion: who even writes subtitles? It feels like a thankless job to write subtitles for rips of movies and TV shows old and new. I get that maybe they source original from, say, Netflix, but .set files existed long before Netflix, and for lots of movies not on Netflix. Writing all those tags for hearing impaired seems like a lot of work, which anyone other than the f…

Thankfully are jobs will be getting easier as speech recognition continues to improve. Some people start from scratch, some people find existing hard subs and translate them to soft subs, some people just translate subs, etc... some people are prolific but the reason it seems most media has a sub is that there are millions of interested and capable people out there, so if a movie is only meaningful enough to only one person out of say 40 million, that's still one person who will make subs for that movie. We have the countless masses to thank, even when sometimes the quality isn't perfect.

Re: Show HN: Automatically synchronize subtitles with video

#72

Post by gsich »

hendry wrote: post #65

What about syncing up audio? Yet to find a CLI solution for syncing up & replacing external audio on my camera's MP4. Current solution is boot up FCPX and do a sync there which rips away minutes of my life.

I use Audacity for this.

I load both files, look for a peak or end of silence in both. The difference I remux with Mkvtoolnix. I do this with 2-3 spots.

Loading the file is usually the longest waiting period.

Re: Show HN: Automatically synchronize subtitles with video

#73

Post by yesenadam »

aabeshou wrote: post #55
hosay123 wrote: post #48

Earlier quoted context omitted.

I thought I understood FFT, but totally not getting the relationship to this problem. Someone ELI5 please? :)

ELI5: You can turn this problem into finding the best "convolution index", and fourier transforms make computing convolutions cheaper. ELIUndergrad: (note that \* means multiplication, there doesnt seem to be a way to escape an asterisk) Lets start by seeing how this is a convolution. We have the videoSpeech sequence, and the subtitle sequence - each is a vector, indexed by time, of 0's and 1's indicating whether the…

Why not use good ol 'x' for multiplication? :-)

Re: Show HN: Automatically synchronize subtitles with video

#74

Post by gsich »

loeg wrote: post #34

> I have yet to find a case where the automatic synchronization has been off by more than ~1 second That's actually quite bad synchronization error for A-V; maybe it's less noticeable for subtitles. I wonder if there is a good way to reduce that error down to milliseconds.

100 ms A/V is already quite noticeable, with some training one might spot 50 ms too.

Re: Show HN: Automatically synchronize subtitles with video

#75

Post by satori99 »

jakecopp wrote: post #64
satori99 wrote: post #3

This is cool. I am constantly impressed by ffmpeg and it capabilities. Last week I was able to make it act as a proxy for live streaming video that reduced the volume of ad-breaks by 50% automatically by listening for SCTE35 cue packets in the stream and adjusting a volume filter accordingly. My housemate has a reality TV addiction, and the ads were getting on my nerves. I just intended to see if it was possible, but…

This sounds amazing! Did you use a USB TV tuner or an online source? Is there chance you could open source the code?

I used an online stream source; A HLS stream that one of my local TV stations broadcasts online.

My method will only work with streams that have SCTE35 packets embedded. This is normally used for local ad insertion on cable networks, so it may not always be available on OTA streams.

There is not much code because it is mostly ffmpeg that does the work.

I used nodejs to manage an ffmpeg child process, read its output streams (one stream over a unix domain socket for the video, and the stdout stream for the SCTE35 packets) and pipe them to a HTTP client.

My code parses the SCTE packets and then sends commands into the running ffmpeg child process to change the audio levels.

I am happy to share it when I get home in a few hours.

Re: Show HN: Automatically synchronize subtitles with video

#76

Post by yesenadam »

kumarharsh wrote: post #59

Completely tangential question to this awesome discussion: who even writes subtitles? It feels like a thankless job to write subtitles for rips of movies and TV shows old and new. I get that maybe they source original from, say, Netflix, but .set files existed long before Netflix, and for lots of movies not on Netflix. Writing all those tags for hearing impaired seems like a lot of work, which anyone other than the f…

I don't know who does it. There are some professionals. Professionals do it for when e.g. BBC or SBS shows non-english movies/series I guess. I wish there was a site where you could put money towards english subs for things!! I guess judging the quality wouldn't be so easy etc. Just in the last few days I've had these subtitle-related issues:

- No English subtitles available for S04 of Rita (danish).. So I made English subtitles for E01, by machine translating existing German and Portuguese subs and using the 'original' German for reference. Turned out pretty great, but took an hour or 2. No way I'd do the whole season like that though.

- Season 2 of Blue Moon (in french) has been out for years, no English subtitles anywhere, seems like there never will be. There's a season 3 too I think, not sure.

- Trying to find subtitles for S02 of Overspel (dutch) there's everything available (but in 1 obscure subtitle site, not easy to find) except E07 and E08. I found someone in a forum asking about exactly those 2 subtitles in January 2014! No-one answered...

I'd be happy to pay for subtitles for things like that that I really want! The cost would be small, I guess, divided by a few or a few dozen or hundred people around the world. Then they would be available online forever, hopefully. Just need to coordinate the effort. That could be a great website.

Also, I helped a bit with some spanish and english subtitles for spanish-language tv shows on viki.com 6 or 7 years ago, which was a pretty great-seeming community-sourced translation effort, but it disappeared, along with all its subtitles. At least most of the content did, it was all asian stuff last time I looked, I think it must have changed hands.

Re: Show HN: Automatically synchronize subtitles with video

#77

Post by hantusk »

I've had great results with the Descript app which can both transcribe and can export a synchronized subtitle file. https://www.descript.com/

Perhaps you can import an srt file, and have it sync'ed? or perhaps you can export the srt text and let Descript sync it and re-create an .srt file.

Re: Show HN: Automatically synchronize subtitles with video

#78

Post by deivid »

satori99 wrote: post #75
jakecopp wrote: post #64

Earlier quoted context omitted.

This sounds amazing! Did you use a USB TV tuner or an online source? Is there chance you could open source the code?

I used an online stream source; A HLS stream that one of my local TV stations broadcasts online. My method will only work with streams that have SCTE35 packets embedded. This is normally used for local ad insertion on cable networks, so it may not always be available on OTA streams. There is not much code because it is mostly ffmpeg that does the work. I used nodejs to manage an ffmpeg child process, read its output…

please do

Re: Show HN: Automatically synchronize subtitles with video

#80

Post by rcthompson »

smacke wrote: post #31
kozikow wrote: post #5

An even harder and cooler problem to solve would be finding extra scenes. Sometimes subtitle or movie have slightly different "cut" (e.g. directors cut) and there are some extra scenes added. A similar algorithmic approach could be used to solve this problem as well.

Totally agreed. I'm not sure how to do the alignment scoring step in this case, perhaps some kind of DTW / FFT hybrid... definitely worth further consideration.

This sounds very similar to the problem of whole-genome alignment in bioinformatics. You have, say, the human and chimp genome sequences, and you want to align them to find out which portions of each genome correspond to which portions in the other, allowing for the possibility of insertions or deletions of DNA sequences in each one.

Unfortunately, I'm not too familiar with the current state of the art for whole-genome alignment, so I don't know for sure which algorithms are considered, but MUMmer4[1] seems like a good place to start. These algorithms are designed to handle sequences up to billions of letters long (e.g. the human genome, which is about 3 billion letters).

[1]: https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

Post reply on HN