Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bypassing YouTube video download throttling

blog.0x7d0.dev

Post reply on HN
Print view View original

31–40 of 242 posts

Re: Bypassing YouTube video download throttling

#31

Post by 1vuio0pswjnm7 »

"Have you ever tried to download videos from YouTube? I mean manually without relying on software like youtube-dl, yt-dlp or one of "these" websites. It's much more complicated than you might think."

This reminds me of some sort of fizzbuzz test. This is not complicated at all. There is no need to use the Range header or run Javascript.

The short script below does not download anything because there is no need. It does not use Range headers, it does not run Javascript and it makes only one TCP connection. With the JSON it fetches, one can simply extract the videoplayback URLs and put them in a locally-hosted HTML page with no Javascript.

    #!/bin/sh
    # usage: echo videoId | $0
For processing the JSON I wrote custom utilities in C that (a) extract videoIds and other useful strings, (b) generate HTTP similar to above, and (c) filter the returned JSON into CSV, SQL or HTML. For me, these run faster than Python and jq and are easier to edit. Using these utilities I can also do full searches that return hundreds to thousands of results and I can easily exclude all "suggested" or "recommended" videos.

CSV output

1666520150,23 Oct 2022 10:15:50 UTC,22,aqz-KE-bpKQ,"Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film",00:10:35,635,UCSMOQeBJ2RAnuFungnQOxLg,19211597,"Blender"

SQL output

INSERT INTO t1(ts,utc,itag,vid,title,dur,len,cid,views,author) VALUES(1666520150,'23 Oct 2022 10:15:50 UTC',22,'aqz-KE-bpKQ','Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film','00:10:35',635,'UCSMOQeBJ2RAnuFungnQOxLg',19211597,'Blender') ON CONFLICT(vid) DO UPDATE SET views=excluded.views;

HTML output

Looks just like CSV except vid is a hyperlink

Re: Bypassing YouTube video download throttling

#32

Post by sph »

benterix wrote: post #23
no_time wrote: post #17

Earlier quoted context omitted.

>Ethically, if you don't only think “fuck Google" Companies do not have ethics, only interests. It's only natural I behave similarly.

> Companies do not have ethics, only interests. While that's certainly true (although an simplification - they are just managed by people with low ethics), we can do better. If I behaved like managers of Google, Meta or Microsoft, I'd be ashamed of myself.

I despise this philosophy. Megacorps are shit, so we should be better than them? It's a popular theme because it makes one feel like they have control over the world around them: they just have to be better than it.

No. I shouldn't have to work harder myself to somehow cancel out the evil in the world. Because if we were less cowards and calling them for what they are - evil - they wouldn't hold so much power over us.

Reminds me of that scene from Mr. Inbetween about bullies. Bullies exist because we're told to ignore them and get punished if we retaliate, so they get away with it.

--

But without getting sidetracked. Companies have no emotion, no particular sense of ethics, and least of all, they don't need people to defend them, unless they're called a lawyer.

Re: Bypassing YouTube video download throttling

#33

Post by mynameisvlad »

G3rn0ti wrote: post #11
mynameisvlad wrote: post #7

Earlier quoted context omitted.

YouTube Premium doesn’t let you download the video file. Even if you do it from a browser, it is only accessible from the YouTube website run offline as a SPA. Even then, downloaded videos can only be played for 29 days before having to reconnect most of the time, with some regions restricting it to 48 hours.

I think parent means the YouTube app. On iOS (and android/chromos?) you can actually download videos and watch them offline if you got a premium membership. But then the videos are under control by the app.

Yes, and I replied to that. It’s not really an alternative to an actual video download.

It’s not the raw video file, you can only access it from the app or website, and there’s restrictions on how long you can be offline before the app/website will stop you from watching it.

Re: Bypassing YouTube video download throttling

#34

Post by wazoox »

0x7d0 wrote: post #2

Have you ever tried to download videos from YouTube? I mean manually without relying on software like youtube-dl, yt-dlp or one of “these” websites. It’s much more complicated than you might think.

A very long time ago, I've worked on a Perl script to do just that ( https://www.perlmonks.org/?node_id=636777 ). Of course the problem with this sort of scripts is that they keep chasing behind changes youtube makes precisely to prevent video downloading.

Re: Bypassing YouTube video download throttling

#35

Post by Applejinx »

planede wrote: post #12
jgtrosh wrote: post #10

Technically, all interesting. Ethically, if you don't only think “fuck Google”, I feel like it's reasonable to stop after the first optimization (“pass the real browser test to get regular browser speeds”). There you're not “wasting” any more of YouTube's resources than a browser user with ad-block. Getting full Gb/s without paying anything feels to me like you're pushing all the ad-blocked users' luck. But then agai…

I wonder how abusive this is though. In the end you are downloading the same amount of data, but in a shorter time. You are utilizing more bandwidth but you go away earlier. I think the original browser use case is tuned for the common occurrence of not watching the whole video. But if you intend to watch (and archive) the whole video to begin with then I don't think this eats away Google's bandwidth more. OTOH it pr…

Well, you go away unless you're another organization that proposes to mirror all of YouTube and eat its lunch.

Google can't only think about humans, it's got to also think about competing organizations. That complicates things.

Re: Bypassing YouTube video download throttling

#36

Post by reggegg »

albert_e wrote: post #18

Some videos offer multiple audio channels for different languages? Why have I never come across such videos before / missed somehow?

Mr Beast's videos have at least a spanish audio track, which funnily enough NewPipe defaults to (or did the last I checked) and NewPipe doesn't support changing the track as far as I can tell

Re: Bypassing YouTube video download throttling

#37

Post by Applejinx »

abwizz wrote: post #22
planede wrote: post #12

Earlier quoted context omitted.

I wonder how abusive this is though. In the end you are downloading the same amount of data, but in a shorter time. You are utilizing more bandwidth but you go away earlier. I think the original browser use case is tuned for the common occurrence of not watching the whole video. But if you intend to watch (and archive) the whole video to begin with then I don't think this eats away Google's bandwidth more. OTOH it pr…

yea, i also don't think that the increased rate is a problem per se, but i also doubt that a majority of youtube views cover the entire length of the video, hence downloaders do probably use more data.

Any youtuber, myself included, can attest that a majority of youtube views demonstrably don't. A key factor in finding success as a youtuber is getting better at retention, hence all the ridiculousness and MrBeastness: some people specialize in retention, and they do better.

It's weird from the standpoint of someone who sets out to watch an entire thing, but almost nobody sticks around while watching videos. It seems like the mass of youtube viewerdom are bouncing around like mad, all the time.

Re: Bypassing YouTube video download throttling

#38

Post by Eavolution »

albert_e wrote: post #18

Some videos offer multiple audio channels for different languages? Why have I never come across such videos before / missed somehow?

Chubbyemu videos often have it, and James Hoffman once used it to provide an audio channel with slurping sounds (he's a coffee channel, that's not as dodgy as it sounds) and one without.

Re: Bypassing YouTube video download throttling

#39

Post by abwizz »

Applejinx wrote: post #37
abwizz wrote: post #22

Earlier quoted context omitted.

yea, i also don't think that the increased rate is a problem per se, but i also doubt that a majority of youtube views cover the entire length of the video, hence downloaders do probably use more data.

Any youtuber, myself included, can attest that a majority of youtube views demonstrably don't. A key factor in finding success as a youtuber is getting better at retention, hence all the ridiculousness and MrBeastness: some people specialize in retention, and they do better. It's weird from the standpoint of someone who sets out to watch an entire thing, but almost nobody sticks around while watching videos. It seems…

not unlike "zapping" tv channels

Re: Bypassing YouTube video download throttling

#40

Post by Gigachad »

no_time wrote: post #19

I'm constantly suprised when YT deploys another half measure against downloaders when GOOG also owns widevine. I wonder what is their reasons for not using it.

Maybe they care about not cutting off devices that don’t support it. TVs, ARM Linux, etc. While making downloading videos just annoying enough that people don’t bother.
Post reply on HN