Live data from Hacker News

YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

twitter.com

21–30 of 166 posts

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#21
This is super cool.

Some of the stuff is kind of questionable to me in the sense that I could believe you could probably make some kind of sufficiently wonky JS that this would do the "wrong" thing.

But it's super cool that they are able to do this as I think it shows that claims of JS complexity based on the size of JS engines is overlooking just how much of that size/complexity comes from the "make it fast" drive vs. what the language requires. Here you have a Mad props to them for even attempting it as well - it simply would not have ever occurred to me to say "let's just write a small JS engine" and I would have spent stupid amounts of time attempting to use JSC* from python instead.

[* JSC appears to be the only JS engine with a pure C API, and the API and ABI are stable so on iOS/macOS at least you can just use the system one which reduces binary size+build annoyance. The downside is that C is terrible, and C++ (differently terrible? :D) APIs make for much more pleasant interfaces to the VM - constructors+destructors mean that you get automatic lifetime management so handles to objects aren't miserable, you can have templates that allow your API to provide handles that have real type information. JSC only has JSValueRef and JSObjectRef, and as a JSObjectRef is a JSValueRef it's actually just a typedef to const JSValueRef :D OTOH other hand I do thing JSC's partially conservative GC is better for stack/temporary variables is superior to Handles for the most part, but it's also absolutely necessary to have an API that isn't absolutely wretched. The real problem with JSC's API is that it has not got any love for many many many .... many years so it doesn't have any way to handle or interact with many modern features without some kludgy wrappers where you push your API objects into JS and have the JS code wrap them up. The API objects are also super slow, as they basically get treated as "oh ffs" objects that obey no rules. I really do wish it would get updated to something more pleasant and really usable.]

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#22
To be clear, this is an extremely tiny subset of JS. It looks like they only implemented the features needed to run a very specific function. For example, the only symbol allowed after "new" is "Date", everything else throws an exception.

It's still fun that it's there, but it's not as big a deal as it sounds from the tweet.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#23
post #19

Nowadays "javascript" refers to the scriptable, grotesquely and absurdely complex and massive web engines, aka google financed blink and geeko, then apple financed webkit, that with their SDK. The currently obfuscated javascript media players will try to break yt-dlp by leveraging the complexity and size of those scripted web engines. They will make them out of reach to small teamns or individuals and it is even "bet…

> but seems to stay in the "reasonable" realm

> M. Bellard and friends

Chose one, that dude is a wizard wielding c like a brain surgeon wields a scalpel.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#24
post #19

Nowadays "javascript" refers to the scriptable, grotesquely and absurdely complex and massive web engines, aka google financed blink and geeko, then apple financed webkit, that with their SDK. The currently obfuscated javascript media players will try to break yt-dlp by leveraging the complexity and size of those scripted web engines. They will make them out of reach to small teamns or individuals and it is even "bet…

Yeah I agree with almost all of this - the massive size and complexity of commercial engines makes it seem like JS the language must also be complex.

I also agree with the idea that these sites will probably be able to/want to create JS that breaks these small/lightweight engines requiring constant work :-/

This final point I disagree with entirely. You can't point to Bellard doing something as evidence that it's reasonable. This is a guy that wrote a program that generated a TV signal via a VGA card. :D

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#26

To be clear, this is an extremely tiny subset of JS. It looks like they only implemented the features needed to run a very specific function. For example, the only symbol allowed after "new" is "Date", everything else throws an exception. It's still fun that it's there, but it's not as big a deal as it sounds from the tweet.

It will only grow - as new scripts will need to be interpreted, new features will be added.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#27

To be clear, this is an extremely tiny subset of JS. It looks like they only implemented the features needed to run a very specific function. For example, the only symbol allowed after "new" is "Date", everything else throws an exception. It's still fun that it's there, but it's not as big a deal as it sounds from the tweet.

if it's going to need much more than that then it probably would make more sense to port the whole application to javascript instead.

but then this could be turned into a commandline browser that is able to interpret a whole web-page and save the resulting html structure instead of the source as curl/wget would do.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#28

Earlier quoted context omitted.

> I believe YouTube limits your bitrate if you don't pass a specific calculated value It's starting to become Widevine bullshit all over again.

It's their platform. They can do with it what they want.

It's their platform but it's also a web site and that comes with certain expectations of interoperability.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#29
post #7
post #2

What purpose does it serve?

I'd have to read up on the specifics as well, but I think basically Youtube uses a lot of obfuscated, rapidly and automatically changing Javascript code to fetch the video data. A project like youtube-dl has to run this code to be able to download videos, because that's what's happening in the browser as well.

What is interesting is it seems to be constant cat and mouse. I download a YT vid. It crawls. Update yt-dlp, it flies again. I love yt-dlp and use it a lot.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#30
post #26

To be clear, this is an extremely tiny subset of JS. It looks like they only implemented the features needed to run a very specific function. For example, the only symbol allowed after "new" is "Date", everything else throws an exception. It's still fun that it's there, but it's not as big a deal as it sounds from the tweet.

It will only grow - as new scripts will need to be interpreted, new features will be added.

I would be horrified if this grew much further. It's perfectly fine for its current scope, but the architecture would not scale at all to a full interpreter without essentially starting from scratch.
Post reply on HN