Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bypassing YouTube video download throttling

blog.0x7d0.dev

Post reply on HN
Print view View original

231–240 of 242 posts

Re: Bypassing YouTube video download throttling

#231

Post by cryptonector »

midoridensha wrote: post #216
cryptonector wrote: post #203

Earlier quoted context omitted.

> Because if we were less cowards and calling them for what they are - evil - they wouldn't hold so much power over us. three years of covid nonsense -> crickets video download speed throttle -> rage Something is wrong here.

That's because there was a very good reason for that Covid "nonsense" as you put it: reducing infection rates and keeping healthcare systems from collapsing. If you disagree with that, then I'm sorry you aren't living in reality and believe in idiotic conspiracy theories and microchips in vaccines and the like. (That said, there really was some real nonsense, such as certain dumb countries that penalized people for g…

> There's no good-for-society reason behind throttling video download speeds.

Google certainly would disagree. Their argument (I assume) would go something like: if people don't pay for content up front and also don't watch the ads then these services can't exist, therefore if these services existing is better for society than them not existing, then it follows that is good for society.

You might not like it but a great deal of our economy is built on that premise, so a lot of people and companies have a stake in holding onto such arguments.

Such an argument is a serious and honest one, and should be responded to with some care. Outright dismissal is not interesting.

> That's because there was a very good reason for that Covid "nonsense" as you put it: reducing infection rates and keeping healthcare systems from collapsing.

That definitely turns out to have been false. The models were vastly wrong. We definitely have differential handling of the pandemic, from Sweden, Africa, and some states in the U.S. for example, and those that went all out did not do better than those that didn't.

Many people warned that this was overblown, but also many people greatly enjoyed exercising authority, and others greatly enjoyed a sense of moral virtue ("saving grandma") that was unjustified.

Re: Bypassing YouTube video download throttling

#232

Post by rasz »

Andrews54757 wrote: post #205
irrational wrote: post #163

Earlier quoted context omitted.

Can you make one for Firefox too? ;-)

it's a simple script, should be portable to firefox with minor modifications. The source code is available on Github [0]. 0: https://github.com/Andrews54757/Anti-Anti-Debug

This is great. I was wondering why does it disable console.table, but then I went to one of my favorite shady sites I screw around with from time to time (I intercept and patch setTimeout/setInterval to disable particular function doing constant debug() injection), lo and behold tables dates and big arrays (not yet covered by your tool) spamming in console :o slowing down devtools window to a crawl.

For console.clear I have this

    window.console.clear = function clear() {console.log(clear.caller, "Asshole script called console.clear()")}
hoping I get a glimpse of what exactly is calling it.

Re: Bypassing YouTube video download throttling

#233

Post by rasz »

0x7d0 wrote: post #2

Have you ever tried to download videos from YouTube? I mean manually without relying on software like youtube-dl, yt-dlp or one of “these” websites. It’s much more complicated than you might think.

Yes, by injecting my own userscript using my (judging by WEI not for long) USER AGENT. I dont even screw around reimplementing their signature/n decoding/throttling functions, I grep for player.js match(/(?:player\/([a-zA-Z0-9_-]+)\/)?(?:html5player|(?:www|player(?:_ias)?))[-\.]([^/]+?)(?:(?:\/html5player(?:-new)?)?|(?:\/[a-z]{2,3}_[A-Z]{2})?\/base)\.js/), then grep in that for relevant functions and call those directly. You could say that from YT perspective everything is 100% kosher, its their own DRM functions unlocking .mp4 link for me :)

Re: Bypassing YouTube video download throttling

#234

Post by 1vuio0pswjnm7 »

1vuio0pswjnm7 wrote: post #218
SuperCuber wrote: post #108

Earlier quoted context omitted.

Interesting. I ran the script to extract the json - that part was almost instant, then i used the first `url` field of `streamingData.adaptiveFormats`. I then ran curl 'https://...googlevideo.com...' --output video.mp4 for me the download is throttled to "768k", i assume thats in bits per second and not bytes which is very low: the random video i tried would take 8 minutes. on the other hand, yt-dlp videoIdHere does…

Is 768k too slow to watch the video from the URL. If not, then I would not call that "throttled". Don't need 500 MB/s to watch a video. From what I've seen, when people discuss YouTube throttling online they are referring to max speeds of 60-70k. That's too slow to watch the video from the URL. Not too slow to download, though. And that's why this idea that YouTube is "preventing" downloads doesn't make any sense. Th…

I should add that with respect to the download URLs in the HTML of every /watch?v= page some will not work at all, namely, in the case of heavily commercialised videos, videos using DASH and some other uncommon cases. But I always found this is minority of linked YouTube videos one encounters on the web.

Re: Bypassing YouTube video download throttling

#235

Post by gloosx »

hurutparittya wrote: post #102
matheusmoreira wrote: post #85

Earlier quoted context omitted.

Honestly recompilation shouldn't even be necessary, browsers should just let us disable the debugger statement when "clever" sites start taking advantage of it. This usage of debuggers to circumvent our tools is abuse and should be literally impossible unless we consent to it. Our computers are our realms. God giveth and god taketh away.

Just wait until you won't even be able to open devtools on a WEI protected website. For your security, of course.

Can't wait when websites start voluntarily cutting themselves out of non-WEI traffic. This means we can take it and serve it, and we will.

Re: Bypassing YouTube video download throttling

#236

Post by gloosx »

dicytea wrote: post #174
probably_wrong wrote: post #28

I've never tried YouTube, but I have downloaded videos from sketchier streaming websites using the web developer tools. Almost all of them have the same protection: some code that triggers only when you open the tools and stops the video by creating a debugger statement you cannot skip and triggering some cpu-heavy code (probably an infinite loop, although I wouldn't discard cryptominers). More importantly this code…

> some code that triggers only when you open the tools I've seen this technique too, but I feel that this is a major flaw on the browser's side. It should be impossible to tell if the dev tools is open or not. Surely this can be done right?

There is no reliable way to tell if they are, so developers have to either listen for sudden page height changes (which usually happens when the dev tool is open, however undocking it fixes the problem) or use one of the few dirty hacks with the Javascript objects, but these are getting patched every year.

Re: Bypassing YouTube video download throttling

#237

Post by bavell »

AltruisticGapHN wrote: post #225
ubercow13 wrote: post #179

Earlier quoted context omitted.

I have noticed that Youtube often re-encodes videos, replacing the data for a certain stream type/number with a new encode.

Yes that is probably what it was! I'll remember that even two+ days after the initial upload, there may still be reencoding. I thought this was only few hours after the creator uploads. So the smaller filesize also is not necessarily a sign of lower quality. In this case it's because it gets a quick first encoding like zip with fast compression, and then it gets reencoded presumably by something much more CPU intensi…

Interesting! Thanks for running mediainfo on them to compare.

When encoding videos there's a "speed" parameter that basically tells the encoder to spend more CPU time compressing the frames. It results in a smaller output size while maintaining the same level of quality but takes much longer to encode. I'm sure that's part of what YT is doing here, an initial quick encode to get the video live then another pass to reduce filesize for long term storage. Good find!

I'm guessing that the perceived higher quality in the smaller file is because the encoder was able to find a more accurate way to represent those frames with the same or fewer number of bits since it has more time to search for the optimal encoding.

Re: Bypassing YouTube video download throttling

#238

Post by glonq »

18dhaFijHH728 wrote: post #195
glonq wrote: post #194

Earlier quoted context omitted.

> On a sidenote am I imagining things or do videos actually look a tiny bit better in YouTube? I'm not [ADVERTISEMENT] sure, because my YouTube viewing [ADVERTISEMENT] experience nowadays is so [ADVERTISEMENT] [ADVERTISEMENT] frequently interrupted with ads that it [ADVERTISEMENT] breaks my focus. #pleaselikeandsubscribeandclickonthenotificationbell

Get sponsor block and ad block extensions or a client with it built in and never think about this again

I block on PC, but on phone and TV I'm stuck with their damn ads.

Re: Bypassing YouTube video download throttling

#239

Post by midoridensha »

userbinator wrote: post #141
hruzgar wrote: post #4

So make a nice, good documented blog post for google engineers to understand and fix this issue?? Whyy

That's what I thought too. Don't give away all your tricks to the enemy; even if the information is already available in other places, the extra publicity won't help our side of the war.

>Don't give away all your tricks to the enemy

That's pretty impossible with open-source software. Google engineers aren't clueless; they'll know where to find this information just as well as anyone here.

It's too bad there isn't a legal way of preventing Google engineers from reading some source code.

Re: Bypassing YouTube video download throttling

#240

Post by AltruisticGapHN »

AltruisticGapHN wrote: post #48

On a sidenote am I imagining things or do videos actually look a tiny bit better in YouTube? This really has puzzled me. I downloaded a few favourites and watch them on VLC or Infuse on my AppleTV. In the YouTube app I can use the "nerd stats" to confirm I am viewing the exact same video/audio streams... ... it could be my imagination but it seems like YouTube does a really subtle kind of filter that makes the "block…

If anyone's still reading this or anyone cares here is what I found out:

- the issue I was experiencing is with a lower quality encoding vp9 stream - from a video that was recently uploaded (explained below)

- though I didn't trust smaller filesize VP9 streams initially, they look in fact noticably better - the picture is smoother, cleaner, the artifacts of compression are less visible. Where AAC can have jittery/glittery distracting dots moving in the background in areas where you have subtle gradients (eg. a plain wall) - Vp9 has none of these, those areas look smoother and cleaner and it gives an overall nicer looking picture without compromising the detail as far I can tell

- Opus audio stream appears to have less of the ReplayGain issue, I'm not sure - but since I downloaded Opus instead of the 140 m4a stream I notice I dont need to adjust the volume compared to viewing same video in YouTube - and since the codec is newer anyway and the filesize is relatively the same or a tad smaller - also it is in 48k not 44k, I am going to download Opus from here on

- a very confusing thing is it appears ; for a recent upload ; you can have an initial VP9 stream of say 500mb which is in fact no better than the AAC and havs the grainy artifacts - and the vp9 stream gets replaced weeks later by one significantly smaller like 400mb vs 500 mb !! and looks way better . whic hsuggst there was a first pass with low quality encoding, replaced by a higher quality encoding later - therefore my assumption that larger filesize is better was wrong

Post reply on HN