Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

twitter.com

Post reply on HN
Print view View original

41–50 of 166 posts

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#41

Post by db48x »

mistrial9 wrote: post #39
db48x wrote: post #17

Earlier quoted context omitted.

It’s not like proof of work at all. It’s just a challenge and response; youtube includes a random number in the webpage for each video, and expects to see a request parameter with a particular value calculated from that random number when you request the video. If you don’t do the arithmetic it throttles you to 50kb/s. Since the calculation of the response is done in JS, and they occasionally change the formula, some…

.. pirate determination is a thing to behold, as is crazed-repetitive digital grabs.. Its not a fair or accurate characterization to dismiss it as "making people's lives harder" .. it is remarkable that the Debian distros now include ytdl; lets do what is reasonable to make it continue

You can’t exactly pirate a youtube video, since they’re all publicly available.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#42

Post by esprehn »

This isn't really JS, it's a purpose built evaluator that's only for evaluating a particular script on YouTube, assuming a huge list of things are true about how YouTube JS is written.

Ex. Its got a hard coded list of methods for String, and it doesn't respect prototypes. It only supports creating Date instances, and won't work if you override the global Date. It parses with regexes and implements all operators with python's operator module (which is the wrong type semantics) etc. Nearly none of the semantics of JS are implemented.

It's sort of the sandwich categorization problem:

If I write a C# "interpreter" in perl thats only 200 lines and just handles string.Join, string.Concat and Console.WriteLine, and it doesn't actually try to implement C# syntax or semantics at all and just uses perl semantics for those operations is it actually C#? :P

I say "not a sandwich".

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#43

Post by esprehn »

olliej wrote: post #21

This is super cool. Some of the stuff is kind of questionable to me in the sense that I could believe you could probably make some kind of sufficiently wonky JS that this would do the "wrong" thing. But it's super cool that they are able to do this as I think it shows that claims of JS complexity based on the size of JS engines is overlooking just how much of that size/complexity comes from the "make it fast" drive v…

This doesn't actually implement any of the JS language though, it just reuses all of python's semantics and hard coded a tiny list of ex. String methods

I also assume you mean mainstream JS engine, but Duktape, JerryScript and QuickJS are all C APIs.

They probably could have used ex. https://github.com/PetterS/quickjs instead of the hacks in the OP linked file.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#44

Post by ajkjk »

rany_ wrote: post #6
anony23 wrote: post #2

What purpose does it serve?

They need to run a JavaScript function to download YouTube videos at normal speeds. Edit: it's also required to download music, otherwise it will just fail Source: - https://github.com/ytdl-org/youtube-dl/issues/29326#issuecom... - https://github.com/ytdl-org/youtube-dl/blob/d619dd712f63aab1... - https://github.com/ytdl-org/youtube-dl/commit/cf001636600430...

Wow:

   Overview of the control flow (already known):
   The Youtube API provides you with n - your video access token
   If their new changes apply to your client (they do for "web") then it is expected your client will modify n based on internal logic. This logic is inside player...base.js
   n is modified by a cryptic function
   Modified n is sent back to server as proof that we're an official client. If you send n unmodified, the server will eventually throttle you.
So they can always change the function to keep you on your toes, hence you need to be able to run semi-arbitrary JS in order to keep using the API.

Waste of human brainpower but I guess that energy is better spent imagining a world where Google isn't in charge instead of kvetching about what they're doing with their influence.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#45

Post by tra3 »

esprehn wrote: post #42

This isn't really JS, it's a purpose built evaluator that's only for evaluating a particular script on YouTube, assuming a huge list of things are true about how YouTube JS is written. Ex. Its got a hard coded list of methods for String, and it doesn't respect prototypes. It only supports creating Date instances, and won't work if you override the global Date. It parses with regexes and implements all operators with…

It’s quacks like a duck at midnight, but it’s actually a frog?

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#46

Post by cheschire »

haunter wrote: post #16

The same in yt-dlp https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/jsinterp... Interesting to see the diffcheck between the two https://www.diffchecker.com/8EJGN27K

Is yt-dlp's implementation being better the reason why I have fewer throttling issues than with youtube-dl?

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#47

Post by MiguelX413 »

uwuemu wrote: post #36
kevin_thibedeau wrote: post #15

Earlier quoted context omitted.

It's their platform. They can do with it what they want.

Quoted post unavailable.

There's a difference between combatting entitlement to a platform and complaining about something not serving a greater good. Leftists are also censored either way. Those companies censor or don't censor according to what would maximize profit. Monetization and reach are probably mutually exclusive with freedom.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#48

Post by Test0129 »

esprehn wrote: post #42

This isn't really JS, it's a purpose built evaluator that's only for evaluating a particular script on YouTube, assuming a huge list of things are true about how YouTube JS is written. Ex. Its got a hard coded list of methods for String, and it doesn't respect prototypes. It only supports creating Date instances, and won't work if you override the global Date. It parses with regexes and implements all operators with…

This really isn't fair. Just because it doesn't faithfully implement whatever standard Javascript is on doesn't mean it isn't an interpreter. All an interpreter is is something that executes a script directly rather than requiring compilation. It is a defacto interpreter for a subset of javascript. Nothing more, nothing less. The title could be more clear, however.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#49

Post by MiguelX413 »

db48x wrote: post #41
mistrial9 wrote: post #39

Earlier quoted context omitted.

.. pirate determination is a thing to behold, as is crazed-repetitive digital grabs.. Its not a fair or accurate characterization to dismiss it as "making people's lives harder" .. it is remarkable that the Debian distros now include ytdl; lets do what is reasonable to make it continue

You can’t exactly pirate a youtube video, since they’re all publicly available.

That's not really how piracy works. I say this as an advocate of it.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#50

Post by tsukikage »

uwuemu wrote: post #36
kevin_thibedeau wrote: post #15

Earlier quoted context omitted.

It's their platform. They can do with it what they want.

Quoted post unavailable.

> you also think billionares should be taxed more

That's... quite a response in defense of a tool intended for breaching TOS and performing copyright infringement. Can you clarify exactly who it is and isn't OK to steal from, again? I'm struggling here.

Post reply on HN