11–20 of 67 posts
Oh, someone has to manually time the movie. They only support about six movies. I expected that it would use closed captioning data, do the sync automatically, and support far more titles.
Oh, someone has to manually time the movie. They only support about six movies. I expected that it would use closed captioning data, do the sync automatically, and support far more titles.
TL;DR: ScreenplaySubs fetches the subtitles from Netflix, parses the PDF-formatted screenplays into JSON, and syncs by calculating the sentence similarities between subtitle and screenplay dialogue.
In particular, we use the Universal Sentence Encoder for deciding whether a subtitle matches with a screenplay dialogue. If a screenplay dialogue is similar enough with the subtitles, the former will be tagged with the timestamp provided by the latter.
A lot of the underlying problems presented with each step sounds deceptively simple at first, but turns out to be quite challenging and fun to research. E.g. Parsing PDFs in general are not straightforward (https://filingdb.com/b/pdf-text-extraction), and there’s only a handful of resources on parsing PDF screenplays beside a handful of research papers (https://github.com/drwiner/ScreenPy/blob/master/INT17_screen...), which lead us to create our own open source repo for this (https://github.com/SMASH-CUT/screenplay-pdf-to-json).
Our screenplay-pdf-to-JSON converter is able to contain all dialogues, transitions, actions within a particular screenplay scene. With this, we’re treating scenes as atomic, being able to detect changes in scene ordering based on the tagged scene timestamps. This also means if dialogues are swapped within a scene in the movie, there will be some syncing inconsistencies.
Some scenes do have little to no dialogues, which would pretty much cause the extension to work on a best-effort basis. E.g. The opening scene of There Will Be Blood has very minimal if not no dialogue at all. This is the case where I need to jump in and sync up the screenplay manually. OTOH, the opening scene of Inglourious Basterds will work very well, since there are tons of dialogues in it. This is the reason why I can’t just add movies and instantly upload it to the site.
Would you be interested for me to get into more details? I was thinking of writing a series of technical blog posts if there are enough interests!
Also check out LanguageLearningWithNetflix [0] which lets you watch videos with two subs in different languages, displays the subs as HTML so select/copy/define will work (and it has a built-in dictionary too). It also allows you to quickly jump to the beginning of each sentence so you can hear it multiple times, which helps improve your listening skills. For me, it has been a fun way to improve my German.
On a side-note, please notice how none of these great features are available to mobile users. iOS for example, is technically perfectly capable of supporting this kind of extensibility, but the App Store model limits it to a few narrow and specific use-cases.
Really cool! It seems like the data synchronization is done manually?
This is great! Also check out LanguageLearningWithNetflix [0] which lets you watch videos with two subs in different languages, displays the subs as HTML so select/copy/define will work (and it has a built-in dictionary too). It also allows you to quickly jump to the beginning of each sentence so you can hear it multiple times, which helps improve your listening skills. For me, it has been a fun way to improve my Ger…
Regarding your comment about extensibility, this applies to mostly every software platform other than the web, unfortunately. And even there, it feels like a happy accident. There's much work to be done in this area.
ScreenplaySubs is a browser extension for Netflix that syncs up movies with screenplays, displaying them side by side. It's like having a subtitle that provides more insights to your films. Demo: https://vimeo.com/447986440
I really like the Amazon Prime Video "X ray" feature that shows the actors, bios, etc, in the current scene. It's odd to me that other services like Netflix don't have something similar.
Oh, someone has to manually time the movie. They only support about six movies. I expected that it would use closed captioning data, do the sync automatically, and support far more titles.
The syncing is done automatically, at least mostly. TL;DR: ScreenplaySubs fetches the subtitles from Netflix, parses the PDF-formatted screenplays into JSON, and syncs by calculating the sentence similarities between subtitle and screenplay dialogue. In particular, we use the Universal Sentence Encoder for deciding whether a subtitle matches with a screenplay dialogue. If a screenplay dialogue is similar enough with…
Anyway I see you have a comment here where you say you use the closed captions to figure out where to staple in the script. Would be cool to be able to staple in arbitrary other media - text audio video whatever.