Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

twitter.com

Post reply on HN
Print view View original

111–120 of 166 posts

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#111

Post by kevingadd »

em-bee wrote: post #75
kevingadd wrote: post #65

Earlier quoted context omitted.

Embedding a whole js engine and then interopping with it from python would be non trivial. Good luck fixing any bugs or corner cases you hit that way. The V8 and spidermonkey embedding apis are both c++ (iirc) and non trivial to use correctly. Having full control like this +simple code is probably lower risk and more maintainable, even if there's the challenge of expanding feature set if scripts change. The alternati…

apparently yt-dlp is somehow calling out to a js engine if available

Yeah, it's possible to install v8 or spidermonkey shells and use them to run code - we use them to run parts of the .NET wasm test suite - but they have a bunch of arbitrary limitations, so if you're trying to emulate a browser I'm not sure I'd bet on them. It's certainly going to be easier than a C++ embedding, so it makes sense that they took that route.

Another option is to use node, but it also has weird limitations/behaviors when running code.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#112

Post by kevingadd »

lloeki wrote: post #104
kevingadd wrote: post #65

Earlier quoted context omitted.

Embedding a whole js engine and then interopping with it from python would be non trivial. Good luck fixing any bugs or corner cases you hit that way. The V8 and spidermonkey embedding apis are both c++ (iirc) and non trivial to use correctly. Having full control like this +simple code is probably lower risk and more maintainable, even if there's the challenge of expanding feature set if scripts change. The alternati…

> Embedding a whole js engine and then interopping with it from python would be non trivial. Cue libv8-node+mini_racer from which PyMiniRacer was born. It is non-trivial but not as hard as one might think. The most painful part is the libv8 build system and Google-centric tooling (depot tools!), which makes it an absolute PITA for libv8 consumers that are not Google/Chrome. This is why the libv8 gem was atrocious to…

Very cool, I'll have to remember that this exists! Looks useful.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#114

Post by Supermancho »

tenebrisalietum wrote: post #38
M30 wrote: post #11

How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?

> How should a programming noob interpret this? The browser is client-facing and everything there is possible to reverse engineer and figure out. So if you design a web-based application, and are depending on client-side Javascript for any security or distribution enforcement, it can be helpful, but can ultimately be unwound and cracked even if obfuscated, etc. > Be impressed at what was achieved here? Yes. Try to do…

Youtube-dl is impressive. This particular hack is not.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#115

Post by chlorion »

M30 wrote: post #11

How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?

The "interpreter" in the youtube-dl source is probably safe from a security standpoint.

yt-dlp seems to support running javascript in a full javascript interpreter/headless browser called phantomjs though. Running javascript in a full interpreter like this is a lot more scary from a security standpoint. I am not sure whether phantomjs sandboxes the javascript evaluation from the rest of the system, and if it does, whether the sandbox actually works properly at all. It looks like the project is not being maintained which is another bad sign.

Big projects with lots of manpower behind them such as chromium have trouble keeping javascript evaluation safe, so I would really suggest not trusting phantomjs on untrusted input.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#116

Post by saagarjha »

btown wrote: post #102
rwmj wrote: post #92

Earlier quoted context omitted.

Google attempting zero days on client computers would be something. It's not totally without precedent (Sony CD rootkits - https://en.wikipedia.org/wiki/Sony_BMG_copy_protection_rootk... ) but would still be major news.

While they likely wouldn't do a zero-day, their JS files, particularly for automated captchas, do push the boundaries of whatever JS engine they're executed inside. See https://github.com/neuroradiology/InsideReCaptcha#the-analys... and note that this analysis is 8 years old. While there's minimal risk if you're either using a full-fledged modern JS engine or a limited-subset interpreter like the OP, an older or non-…

I mean the DOS would be that your youtube-dl invocation hangs, and then you kill it.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#117

Post by LeoPanthera »

cheschire wrote: post #46
haunter wrote: post #16

The same in yt-dlp https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/jsinterp... Interesting to see the diffcheck between the two https://www.diffchecker.com/8EJGN27K

Is yt-dlp's implementation being better the reason why I have fewer throttling issues than with youtube-dl?

Maybe this isn't true anymore, but for a while they would hit different APIs. yt-dlp was using the Android YouTube API because it had no throttling.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#118

Post by derangedHorse »

delusional wrote: post #103

Can we stop the trend of linking to tweets that just contain another link to the content? what's the point? Wouldn't this be 10x better if it was a link directly to the github?

I like the Twitter linking since it's almost like the OP is giving credit to where they found the information from.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#119

Post by caned »

delusional wrote: post #103

Can we stop the trend of linking to tweets that just contain another link to the content? what's the point? Wouldn't this be 10x better if it was a link directly to the github?

I often share links to HN instead of the referred link. Many times the comments are as interesting as the content. This applies to sharing Twitter or Reddit links, too, albeit with a lower S/N ratio.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#120

Post by plaguepilled »

derangedHorse wrote: post #118
delusional wrote: post #103

Can we stop the trend of linking to tweets that just contain another link to the content? what's the point? Wouldn't this be 10x better if it was a link directly to the github?

I like the Twitter linking since it's almost like the OP is giving credit to where they found the information from.

Agreed. If you only know this from someone else's observation, you should link the observation.
Post reply on HN