How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?
YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
11–20 of 166 posts
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#12What purpose does it serve?
I'd have to read up on the specifics as well, but I think basically Youtube uses a lot of obfuscated, rapidly and automatically changing Javascript code to fetch the video data. A project like youtube-dl has to run this code to be able to download videos, because that's what's happening in the browser as well.
For those interested further, in some of the past few weeks youtube-dl had stopped working intermittently for multiple hours at a time, and it was precisely related to this code.
We have a custom-made Discord music bot on our server which uses ytdl to stream songs so we can listen together, and at one point we were listening and suddenly got some obscure JavaScript error.
We began joking that there's some bug in the code which breaks it after 6PM, but later found out that Google had changed some of the obfuscated JS and this basically broke this part of code, which prevented us from fetching the song information.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#13How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?
All of the above, really.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#14Awesome. Even if it's likely incomplete, it might come in really handy for some scraping I need to do...
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#15Earlier quoted context omitted.
It's used in the YouTube extractor: https://github.com/ytdl-org/youtube-dl/blob/d619dd712f63aab1... I believe YouTube limits your bitrate if you don't pass a specific calculated value; it's possible youtube-dl has to parse and eval JS to get it.
> I believe YouTube limits your bitrate if you don't pass a specific calculated value It's starting to become Widevine bullshit all over again.
It's their platform. They can do with it what they want.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#16The same in yt-dlp https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/jsinterp...
Interesting to see the diffcheck between the two https://www.diffchecker.com/8EJGN27K
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#17What purpose does it serve?
You need to run some obscured JS to get decent download speeds from Youtube. Something along the lines of PoW.
It’s not like proof of work at all. It’s just a challenge and response; youtube includes a random number in the webpage for each video, and expects to see a request parameter with a particular value calculated from that random number when you request the video. If you don’t do the arithmetic it throttles you to 50kb/s.
Since the calculation of the response is done in JS, and they occasionally change the formula, some download programs are moving towards running the JS rather than trying to keep up with the changes.
It’s really just bullshit to make people’s lives harder.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#18Earlier quoted context omitted.
> I believe YouTube limits your bitrate if you don't pass a specific calculated value It's starting to become Widevine bullshit all over again.
It's their platform. They can do with it what they want.
it’s sort of an extension of the state / surveillance
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#19Nowadays "javascript" refers to the scriptable, grotesquely and absurdely complex and massive web engines, aka google financed blink and geeko, then apple financed webkit, that with their SDK.
The currently obfuscated javascript media players will try to break yt-dlp by leveraging the complexity and size of those scripted web engines. They will make them out of reach to small teamns or individuals and it is even "better", it will force ppl to use apple or google web engine, killing any attempt to provide a real alternative.
A standalone javascript interpreter is actually some work, but seems to stay in the "reasonable" realm: look at quickjs from M. Bellard and friends (the guy who created qemu, ffmpeg, tinycc, etc): plain and simple C (no need of a c++ compiler), doing the job more that well enough.
That's why noscript/basic (x)html is so much important.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#20How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?
It's an extremely tiny subset of JS—as an example, the only object that can be instantiated is Date. Anything other than "Date" after "new" throws an exception.
It's definitely neat, but not especially useful outside of the confines of its current application, and the security concerns of such a tiny subset will be minimal.