> I have yet to find a case where the automatic synchronization has been off by more than ~1 second.
1 second is in the 0 - 1000ms which is still a lot of room for manual tweaking in VLC. I can notice 50ms differences in sync.
Show HN: Automatically synchronize subtitles with video
81–90 of 129 posts
Re: Show HN: Automatically synchronize subtitles with video
#82Completely tangential question to this awesome discussion: who even writes subtitles? It feels like a thankless job to write subtitles for rips of movies and TV shows old and new. I get that maybe they source original from, say, Netflix, but .set files existed long before Netflix, and for lots of movies not on Netflix. Writing all those tags for hearing impaired seems like a lot of work, which anyone other than the f…
Besides people who manually write subtitles, there's also a whole slew being ripped from DVD's and such.
Re: Show HN: Automatically synchronize subtitles with video
#83Earlier quoted context omitted.
Previously: Fiddle with the subtitle/audio offset factor (and then they drift apart again slowly, driving you mad!) Now: subsync Soon: Players run subsync internally the press of one button or commandline switch. The voice audio detection and then mapping is such a neat solution. I would have embedded parts of the surrounding audio in some base64 format into the subtitle file and then used that as an alignment clue.…
Actually, it unfortunately doesn't work if the drift gets worse over time -- so far, it only works with constant drift. Maybe fixing constant 1st derivative drift is the next step!
The drifts come from different speed of videos compared to the speed of the videos for which the titles were done.
E.g. if one subtitle is made for 24 frames per second speed (classic film speed) and you have a video presented in 25 frames per second (common in Europe). The original two hours video in 24 fps is then 5 minutes shorter in the Europe-origin version. Or the opposite: the subtitles for 120 minutes would at the end appear 5 minutes before!
Apparently there are some other speed changes, for which I don't know how they happen.
I have done one such correction once, using the linear function to model the correction ^based on the target times of the first and of the last title.
Re: Show HN: Automatically synchronize subtitles with video
#84I will love you for this. really. I can't tell how much time I have lost searching for the correct subtitles, in various languages
Re: Show HN: Automatically synchronize subtitles with video
#85Completely tangential question to this awesome discussion: who even writes subtitles? It feels like a thankless job to write subtitles for rips of movies and TV shows old and new. I get that maybe they source original from, say, Netflix, but .set files existed long before Netflix, and for lots of movies not on Netflix. Writing all those tags for hearing impaired seems like a lot of work, which anyone other than the f…
For more obscure movies there are specialized forums where people do that for some bounty (mostly in upload credit) to make it possible for others to enjoy that movie they don’t speak the language of too. Not much different to the question why people spend days ripping music, movies just to share it with others.
Re: Show HN: Automatically synchronize subtitles with video
#86Earlier quoted context omitted.
I thought I understood FFT, but totally not getting the relationship to this problem. Someone ELI5 please? :)
ELI5: You can turn this problem into finding the best "convolution index", and fourier transforms make computing convolutions cheaper. ELIUndergrad: (note that \* means multiplication, there doesnt seem to be a way to escape an asterisk) Lets start by seeing how this is a convolution. We have the videoSpeech sequence, and the subtitle sequence - each is a vector, indexed by time, of 0's and 1's indicating whether the…
> ELI5: You can turn this problem into finding the best "convolution index", and fourier transforms make computing convolutions cheaper.
I tried this sentence on my 5-year-old and got a blank stare. He then proceeded to tell me a story about how Darth Vader is so scary and cool and that he's actually Luke's father. YMMV.
Re: Show HN: Automatically synchronize subtitles with video
#87Earlier quoted context omitted.
ELI5: You can turn this problem into finding the best "convolution index", and fourier transforms make computing convolutions cheaper. ELIUndergrad: (note that \* means multiplication, there doesnt seem to be a way to escape an asterisk) Lets start by seeing how this is a convolution. We have the videoSpeech sequence, and the subtitle sequence - each is a vector, indexed by time, of 0's and 1's indicating whether the…
Why not use good ol 'x' for multiplication? :-)
[deleted]
Re: Show HN: Automatically synchronize subtitles with video
#88> Instead, we use the fact that "scoring all alignments" is a convolution operation and can be implemented with the Fast Fourier Transform (FFT), bringing the complexity down to O(n log n). Not sure I understand this right - is this basically treating both binary strings as square waves, converting them to the frequency domain and determining the offset as a pitch shift between the two spectrograms?
That is a very 'signals' based interpretation of the convolution theorem [1] applied to binary functions.
In essence, the fourier-transform is based on convolutions. F(eta) is essentially the convolution of f(x) with sin(eta x).* This is very loosely why the convolution theorem works.
[1] https://en.wikipedia.org/wiki/Convolution_theorem
* This excludes all cosine parts of the transform. Its neater to work in the complex domain and state that:
F(eta) = f(x) convolved with e^(eta i x)
Re: Show HN: Automatically synchronize subtitles with video
#89Earlier quoted context omitted.
ELI5: You can turn this problem into finding the best "convolution index", and fourier transforms make computing convolutions cheaper. ELIUndergrad: (note that \* means multiplication, there doesnt seem to be a way to escape an asterisk) Lets start by seeing how this is a convolution. We have the videoSpeech sequence, and the subtitle sequence - each is a vector, indexed by time, of 0's and 1's indicating whether the…
Why not use good ol 'x' for multiplication? :-)
'x' is a letter, not the times symbol.
Re: Show HN: Automatically synchronize subtitles with video
#90This blows my mind (like everyone else's here)! While playing digital copies of somewhat older movies (before BluRay rips came into vogue), a problem which surfaces frequently is that the frame rates of the video and the subtitle track are slightly mismatched, say video at 24fps, subtitle at 25.6fps(?). This makes the subtitles drift away from the video and require a manual intervention every few minutes. If I can't…
As others have said, matching cuts is very similar to genome matching. In genome matching, often genes are slightly offset, so there is need to correct for that.
Matching speed might be possible based on first matching runs of one minute, and then trying to scale with that minute as a fixed point to see which scale works out. Perhaps some heuristics to take common frame rate-ratios into account might help.
An even crazier idea:
Use the fact that scaling becomes shifting after a logarithm, and try to match after setting g(x) = f(log(x)).