TIL that you can run frontend Javascript with a package like Deno. I thought you need a proper headless browser for it.
Yt-dlp: Upcoming new requirements for YouTube downloads
131–140 of 635 posts
Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#132Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code. PoToken - Proof…
If you ever wondered why the likes of Google and Cloudflare want to restrict the web to a few signed, integrity-checked browser implementations? Now you know.
The web as we knew it before ChatGPT was built around the idea that humans have to scavenge for information, and while they're doing that, you can show them ads. In that world, content didn't need to be too protected because you were making up for it in eyeballs anyway.
With AI, that model is breaking down. We're seeing a shift towards bot traffic rather than human traffic, and information can be accessed far more effectively and, most importantly, without ad impressions. So, it makes total sense for them to be more protective about who has access to their content and to make sure people are actually paying for it, be it with ad views or some other form of agreement.
Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#133Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code. PoToken - Proof…
> For the web it requires that you run a snippet of javascript code (the challenge) in the browser to prove that you are not a bot. How does this prove you are not a bot. How does this code not work in a headless Chromimum if it's just client side JS?
I have a little experience with Selenium headless on Facebook. Facebook tests fonts, SVG rendering, CSS support, screen resolution, clock and geographical settings, and hundreds of other things that give it a very good idea of whether it's a normal client or Selenium headless. Since it picks a certain number of checks more or less at random and they can modify the JS each time it loads, it is very, very complicated to simulate.
Facebook and Instagram know this and allow it below a certain limit because it is more about bot protection than content protection.
This is the case when you have a real web browser running in the background. Here we are talking about standalone software written in Python.
Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#134It's almost funny, not to mention sad, that their player/page has been changed, filling it with tons of JS that makes less powerful machines lag. For a while now, I've been forced to change "watch?v=" to "/embed/" to watch something in 480p on an i3 Gen 4, where the same video, when downloaded, uses ~3% of the CPU. However, unfortunately, it doesn't always work anymore. https://www.youtube.com/watch?v=xvFZjo5PgG0 htt…
Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#135Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#136Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#137At some point we’re going to need a better place to put videos than YouTube. The lack of any democratization of bulk storage is beginning to be a real problem on the internet. Yes, we have archive.org. We need more than that, though. I’m sure there’s some distributed solution like IPFS but I haven’t seen any serious attempt to make this accessible to every day people.
> The lack of any democratization of bulk storage is beginning to be a real problem on the internet. There are many thousands of paid hosting services, feel free to pick one. It turns out hosting TB of data for free is a pretty tricky business model to nail down.
And even if the legal attacks could be mitigated most people would still use youtube because they're there for the money (or for people who are there for the money). They are not there for a video host. Youtube enables distribution of money and there's no way that any government would let any free system distribute money without even more intense legal, and indeed physically violent, attacks.
Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#138Earlier quoted context omitted.
If you ever wondered why the likes of Google and Cloudflare want to restrict the web to a few signed, integrity-checked browser implementations? Now you know.
>If you ever wondered why the likes of Google and Cloudflare want to restrict the web I disagree with the framing of "us vs them". It's actually "us vs us". It's not just us plebians vs FAANG giants. The small-time independent publishers and creators also want to restrict the web because they don't want their content "stolen". They want to interact with real humans instead of bots. The following are manifestations of…
I'm not in it for the dollar. I just want the licenses I put on my content/code to be respected, that's all. IOW, I don't what I put out there to be free forever (as in speech and beer) to be twisted and monetized by the people who re in this for the dollar.
Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#139Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code. PoToken - Proof…
If you ever wondered why the likes of Google and Cloudflare want to restrict the web to a few signed, integrity-checked browser implementations? Now you know.
I laugh at people who think ActivityPub or Mastodon or BlueSky will save us. We already had that, it was called e-mail, look what happened once everyone started using it.
If we couldn't stop the centralization effects that occurred on e-mail, any attempt to stop centralization in general is honestly a utopian fool's errand. Regulation is easier.
Re: Yt-dlp: Upcoming new requirements for YouTube downloads
#140Why won’t they use my browser for downloads, for example through TestCafe? That would also allow downloading premium quality (for subscribers) and so on.