Live data from Hacker News

Yt-dlp: Upcoming new requirements for YouTube downloads

github.com

131–140 of 635 posts

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#131

TIL that you can run frontend Javascript with a package like Deno. I thought you need a proper headless browser for it.

I think you only need something like `jsdom` to have the core API available. The DOM itself is just a tree structure with special nodes. Most APIs are optional and you can provide stubs if you're targeting a specific websites. It's not POSIX level.

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#132

Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code. PoToken - Proof…

If you ever wondered why the likes of Google and Cloudflare want to restrict the web to a few signed, integrity-checked browser implementations? Now you know.

I don't know, it's really hard to blame them. In a way, the next couple of years are going to be a battle to balance easy access to info with compensation for content creators.

The web as we knew it before ChatGPT was built around the idea that humans have to scavenge for information, and while they're doing that, you can show them ads. In that world, content didn't need to be too protected because you were making up for it in eyeballs anyway.

With AI, that model is breaking down. We're seeing a shift towards bot traffic rather than human traffic, and information can be accessed far more effectively and, most importantly, without ad impressions. So, it makes total sense for them to be more protective about who has access to their content and to make sure people are actually paying for it, be it with ad views or some other form of agreement.

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#133

Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code. PoToken - Proof…

> For the web it requires that you run a snippet of javascript code (the challenge) in the browser to prove that you are not a bot. How does this prove you are not a bot. How does this code not work in a headless Chromimum if it's just client side JS?

Once JavaScript is running, it can perform complex fingerprinting operations that are difficult to circumvent effectively.

I have a little experience with Selenium headless on Facebook. Facebook tests fonts, SVG rendering, CSS support, screen resolution, clock and geographical settings, and hundreds of other things that give it a very good idea of whether it's a normal client or Selenium headless. Since it picks a certain number of checks more or less at random and they can modify the JS each time it loads, it is very, very complicated to simulate.

Facebook and Instagram know this and allow it below a certain limit because it is more about bot protection than content protection.

This is the case when you have a real web browser running in the background. Here we are talking about standalone software written in Python.

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#134

It's almost funny, not to mention sad, that their player/page has been changed, filling it with tons of JS that makes less powerful machines lag. For a while now, I've been forced to change "watch?v=" to "/embed/" to watch something in 480p on an i3 Gen 4, where the same video, when downloaded, uses ~3% of the CPU. However, unfortunately, it doesn't always work anymore. https://www.youtube.com/watch?v=xvFZjo5PgG0 htt…

Personally I am looking to get away from Youtube and looking towards some form of PeerTube/peer-based platform.

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#136
Good to see the mice are still winning the cat-and-mouse game. Selfishly, I kind of want the cat to start to win, to satisfy my curiosity. I predict that if YouTube ever actually blocked downloading, a YouTube competitor that supports downloading would start to immediately gain popularity. I want to know if I'm right about that, and there's no way to test unless Google actually starts to win. Go, Google, go! I believe in you!

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#137
post #45

At some point we’re going to need a better place to put videos than YouTube. The lack of any democratization of bulk storage is beginning to be a real problem on the internet. Yes, we have archive.org. We need more than that, though. I’m sure there’s some distributed solution like IPFS but I haven’t seen any serious attempt to make this accessible to every day people.

> The lack of any democratization of bulk storage is beginning to be a real problem on the internet. There are many thousands of paid hosting services, feel free to pick one. It turns out hosting TB of data for free is a pretty tricky business model to nail down.

There have been plenty of free distributed hosting services for the web that worked perfectly (popcorn time, etc, etc). It's just that every time they become popular they are attacked legally and shut down. The problem is not technical, or even resource based, the problem is legal. Only a mega-corp can withstand the legal attacks.

And even if the legal attacks could be mitigated most people would still use youtube because they're there for the money (or for people who are there for the money). They are not there for a video host. Youtube enables distribution of money and there's no way that any government would let any free system distribute money without even more intense legal, and indeed physically violent, attacks.

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#138
post #99

Earlier quoted context omitted.

If you ever wondered why the likes of Google and Cloudflare want to restrict the web to a few signed, integrity-checked browser implementations? Now you know.

>If you ever wondered why the likes of Google and Cloudflare want to restrict the web I disagree with the framing of "us vs them". It's actually "us vs us". It's not just us plebians vs FAANG giants. The small-time independent publishers and creators also want to restrict the web because they don't want their content "stolen". They want to interact with real humans instead of bots. The following are manifestations of…

> Now everybody on the globe wants to try to make a dollar, and likewise, they feel dollars are being stolen from them.

I'm not in it for the dollar. I just want the licenses I put on my content/code to be respected, that's all. IOW, I don't what I put out there to be free forever (as in speech and beer) to be twisted and monetized by the people who re in this for the dollar.

Re: Yt-dlp: Upcoming new requirements for YouTube downloads

#139

Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code. PoToken - Proof…

If you ever wondered why the likes of Google and Cloudflare want to restrict the web to a few signed, integrity-checked browser implementations? Now you know.

Everything trends towards centralization on a long enough period.

I laugh at people who think ActivityPub or Mastodon or BlueSky will save us. We already had that, it was called e-mail, look what happened once everyone started using it.

If we couldn't stop the centralization effects that occurred on e-mail, any attempt to stop centralization in general is honestly a utopian fool's errand. Regulation is easier.

Post reply on HN