I do wonder why YouTube does not try harder to make it difficult to do this computation meant to prove you are a legit YouTube web client. Providing an easy-to-find, simple JS function interpretable with 900 lines of Python is like they don't try at all. They might as well do nothing. Or is their goal just to make youtube-dl not 100% reliable? Or to be able to say "look, you are running our code in a way we did not i…
YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
91–100 of 166 posts
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#92Earlier quoted context omitted.
And as a user of youtube-dl, I'm quite happy about this. This probably allows a very safe, restricted "subset" of JS. Way better than using a full JS engine. 900 lines is still small and manageable.
I'm trying to get the thread model here. Is the concern that Youtube will inject JS into the payload which tries to break out of the youtuble-dl js sandbox using some zero day in whatever js engine they would use instead?
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#93Earlier quoted context omitted.
And as a user of youtube-dl, I'm quite happy about this. This probably allows a very safe, restricted "subset" of JS. Way better than using a full JS engine. 900 lines is still small and manageable.
I'm trying to get the thread model here. Is the concern that Youtube will inject JS into the payload which tries to break out of the youtuble-dl js sandbox using some zero day in whatever js engine they would use instead?
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#94How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?
It is technically wrong - it isn't a sufficiently rich and powerful approach to handle all JS (HTML) that you might throw at it. It'll work for a while until it eventually barfs when you least expect it.
EXCEPT that if the inputs you are giving it come from some understood source(s) that aren't likely to change, then a simpler approach to the "all singing all dancing" correct may be appropriate and justified. E.g. because it might be easier to write, easier to maintain and/or less attack surface etc.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#95Anyone who has ever pulled a website from a script knows the pain that is Javascript. Normally you want to just get some text and work out the API actions but a lot of sites use horribly obfuscated Javascript -- either because that's what modern web development is (lolz) -- or because its part of their 'security.' That means if you want to write browser-based bots properly -- you ought to use a browser. There are spe…
Just npm install puppeteer.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#96Earlier quoted context omitted.
And as a user of youtube-dl, I'm quite happy about this. This probably allows a very safe, restricted "subset" of JS. Way better than using a full JS engine. 900 lines is still small and manageable.
I'm trying to get the thread model here. Is the concern that Youtube will inject JS into the payload which tries to break out of the youtuble-dl js sandbox using some zero day in whatever js engine they would use instead?
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#97Earlier quoted context omitted.
There's a huge difference between an interpreter for "JavaScript" and an interpreter for a "subset of JavaScript".
Making a pedantic argument on what constitutes an interpreter is silly. The title is bad. It is an interpreter. I'll continue to eat downvotes on this because of the pedantry of HN.
most of us know that a thousand or so lines of code is not a full JavaScript interpreter and cannot be the real thing.
there is no argument or conversation to have about it.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#98How should a programming noob interpret this? Be impressed at what was achieved here? Be concerned about security implications using the tool? Something else entirely?
This isn't something YouTube particularly enjoys. They would rather you keep coming back -- every visit is more ad revenue for them. If you have an offline copy, you don't need to visit YouTube anymore.
YouTube has an incentive, therefore, to make it more difficult to download (or "scrape") their content.
I'm not particularly sure of the specific details, but apparently YouTube has added JavaScript (a programming language that executes in the browser) as a hurdle to jump over. A simple python script doesn't have enough brains to execute JavaScript, only enough to realize that it exists. (Clearly, youtube-dl is sophistication enough to have jumped over it.)
These are the conclusions I come to, having written software for about a decade.
1) Once you give information to someone, be it text, pictures, sound, or video -- they will do whatever they want with it, and you have no control. Oh, yes -- it may be illegal. Maybe unethical. But the fact of the matter is you do not have control over information once it leaves your hands.
2) Adding hurdles to make it harder to access the information does little to stop someone who is dedicated to accessing it.
3) Implementing a subset of JavaScript in such an elegant and tiny manner is quite impressive.
How you interpret these facts depends on your worldviews. If you are a media and content creator, you will view these facts differently than a politician, and a teenager.
As an engineer and amateur philosopher, I certainly support the rights of content creators to be paid for their work. And yet, I fear that more and more, content creators want to lease me a right to listen their music, instead of own a copy of it.
I used to own CDs, DVDs, movies, and books. What happens if Amazon or YouTube decides to not serve me anymore? Anything I've "purchased" from them, I lose access to.
Further more, if I create a song, I used to be able to burn copies of CDs and distribute it on the street corners. Now, you have to sign up to stream on Spotify. This is a double edged sword -- I get a wide audience, but Spotify will do whatever they want with me.
This troubles me.
Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#99Re: YouTube-dl has an interpreter for a subset of JavaScript in 870 lines of Python
#100Earlier quoted context omitted.
And as a user of youtube-dl, I'm quite happy about this. This probably allows a very safe, restricted "subset" of JS. Way better than using a full JS engine. 900 lines is still small and manageable.
I'm trying to get the thread model here. Is the concern that Youtube will inject JS into the payload which tries to break out of the youtuble-dl js sandbox using some zero day in whatever js engine they would use instead?