Live data from Hacker News

YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

twitter.com

131–140 of 166 posts

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#131
post #11

How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?

In the face of weird shit like this, I give you the permission to go with your gut.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#132

Earlier quoted context omitted.

I would be horrified if this grew much further. It's perfectly fine for its current scope, but the architecture would not scale at all to a full interpreter without essentially starting from scratch.

How much does it need to "scale"? It just has to be fast and correct enough for the a CLI to work.

"Scale" has multiple meanings, and here I'm not talking about speed.

This interpreter is built around matching specific Regex patterns and then immediately executing hard-coded behaviors with a few slots for parameters. It's missing a whole lot of the skeletal structure that would be necessary to "scale" it up to support a generally useful subset of JS, much less the entire language. Without the necessary structure, it would be a buggy mess that's impossible to maintain, and you can't just take the existing code and structure it better: it's built on the wrong foundation. That's what I mean when I say it won't scale.

That's not a judgement against the team that wrote it! It meets their needs fine, and choosing a minimal solution is great engineering that takes a lot of guts in the current software culture. I like what they've done here. Just don't take it and try to scale it up into a full JS runtime.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#133
post #121

Earlier quoted context omitted.

Agreed. If you only know this from someone else's observation, you should link the observation.

That is against HN guidelines: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html

Citing the guidelines is against the guidelines, if not by the letter, in spirit. It's boring and it lacks curiosity. It assumes too much about the sharer. "Can we stop this trend" is a dog whistle for the "HN is getting worse" complaint.

Instead we could be considering if we're meant to read the Twitter conversation as well, or sharing a laugh about the link in the tweet author's bio. Or maybe the sharer didn't feel comfortable enough seeming like they made the claim but still wanted to share it because it's kind of cool.

AFAIK there's no junior HN mod of the year award.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#134
post #121

Earlier quoted context omitted.

Agreed. If you only know this from someone else's observation, you should link the observation.

That is against HN guidelines: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html

I'm in stitches that my other comment was flagged at 0 min. Don't ever change, you guys ;-)

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#135

Anyone who has ever pulled a website from a script knows the pain that is Javascript. Normally you want to just get some text and work out the API actions but a lot of sites use horribly obfuscated Javascript -- either because that's what modern web development is (lolz) -- or because its part of their 'security.' That means if you want to write browser-based bots properly -- you ought to use a browser. There are spe…

Embedding V8 can work quite well: https://github.com/sqreen/PyMiniRacer

You probably have to emulate some of the DOM, but you can interact directly with whatever obfuscated/packed scripts in a more lightweight and secure way than driving an entire browser.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#136

Anyone who has ever pulled a website from a script knows the pain that is Javascript. Normally you want to just get some text and work out the API actions but a lot of sites use horribly obfuscated Javascript -- either because that's what modern web development is (lolz) -- or because its part of their 'security.' That means if you want to write browser-based bots properly -- you ought to use a browser. There are spe…

I use pyminiracer to great effect for that sort of scraping.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#138
post #107

Earlier quoted context omitted.

I would be horrified if this grew much further. It's perfectly fine for its current scope, but the architecture would not scale at all to a full interpreter without essentially starting from scratch.

Yeah, at some point you have to question if it's worth spending time maintaining a quirky, error-prone, ever-growing mini-JS interpreter, or just adding a dependency on v8 or node or something. And then you don't have to worry about supporting new scripts, as they'll just always work.

Being pure Python has an advantage where it can be run e.g., in Pythonista 3 for iOS (which allows to implement various functions such as: download just audio, send link to be viewed on a separate device).

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#139
post #19

Nowadays "javascript" refers to the scriptable, grotesquely and absurdely complex and massive web engines, aka google financed blink and geeko, then apple financed webkit, that with their SDK. The currently obfuscated javascript media players will try to break yt-dlp by leveraging the complexity and size of those scripted web engines. They will make them out of reach to small teamns or individuals and it is even "bet…

> quickjs from M. Bellard and friends Is the M key next to the F key on your particular keyboard by chance? Because I've always called him "Fabrice." https://en.wikipedia.org/wiki/Fabrice_Bellard

Could just be the usual abbreviation for Monsieur.

Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python

#140
post #19

Nowadays "javascript" refers to the scriptable, grotesquely and absurdely complex and massive web engines, aka google financed blink and geeko, then apple financed webkit, that with their SDK. The currently obfuscated javascript media players will try to break yt-dlp by leveraging the complexity and size of those scripted web engines. They will make them out of reach to small teamns or individuals and it is even "bet…

> quickjs from M. Bellard and friends Is the M key next to the F key on your particular keyboard by chance? Because I've always called him "Fabrice." https://en.wikipedia.org/wiki/Fabrice_Bellard

Monsieur Bellard - M. Bellard
Post reply on HN