Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yark: Advanced and easy YouTube archiver now stable

github.com

Post reply on HN
Print view View original

101–110 of 148 posts

Re: Yark: Advanced and easy YouTube archiver now stable

#101

Post by romwell »

0cf8612b2e1e wrote: post #58
romwell wrote: post #55

Earlier quoted context omitted.

FSM almighty, how much video are you watching to have that many favorite ones?

With the rise of cheap 4k cameras, it goes faster than you think. I download guitar training videos where a 25 minute tutorial can be 2-3gbs. I believe those are “only” 2k resolution. If you do not download them in a space conscious format or recompress (using mildly aggressive settings can drop them to 100mb), a handful of videos can quickly fill a hard drive.

Why wouldn't one download something like that in 720P video/196kbps audio?

I mean, there's a time and a place for 4K, but watching the zits on the face of a guy who tells you how to play C Am F G on a guitar isn't it.

Not to mention, most of those cheap 4K cameras won't have optics to utilize those pixels; no quality will be lost in 720p.

Re: Yark: Advanced and easy YouTube archiver now stable

#102

Post by kristianp »

romwell wrote: post #101
0cf8612b2e1e wrote: post #58

Earlier quoted context omitted.

With the rise of cheap 4k cameras, it goes faster than you think. I download guitar training videos where a 25 minute tutorial can be 2-3gbs. I believe those are “only” 2k resolution. If you do not download them in a space conscious format or recompress (using mildly aggressive settings can drop them to 100mb), a handful of videos can quickly fill a hard drive.

Why wouldn't one download something like that in 720P video/196kbps audio? I mean, there's a time and a place for 4K, but watching the zits on the face of a guy who tells you how to play C Am F G on a guitar isn't it. Not to mention, most of those cheap 4K cameras won't have optics to utilize those pixels; no quality will be lost in 720p.

That depends, if the source is bright enough, those little pixels will have enough signal to work effectively.

Re: Yark: Advanced and easy YouTube archiver now stable

#103

Post by tcmb »

imiric wrote: post #87
KMnO4 wrote: post #16

Earlier quoted context omitted.

I did some napkin math. If 1 billion people each backed up 10gb, we’d almost have enough to store a copy of YouTube with zero data redundancy. Google is massive.

That's impressive indeed, but if we boil it down to just the things that are worth saving, removing duplicates, long and pointless livestreams, long videos that are just endless loops, etc., it could be done with much less space. If we also ignore auto generated spam[1] and harmful content (Elsagate, etc.), it would require even less. It's a nice thought experiment, but we really don't need to archive all of YT. That…

I would argue that backing up selectively instead of just grabbing everything makes the job harder, not easier. You might need less storage, but who will decide (and how) what gets stored and what doesn't?

Re: Yark: Advanced and easy YouTube archiver now stable

#104

Post by 6510 »

An idea I had some time ago:

BitTorrent magnet uri's or hashes are suppose to be made from torrents but they really just point at a torrent.

One could make torrents for each video, take the youtube url v param and make a hash from that and point it ("erroneously") at the torrent.

That way, provided it was downloaded before, anyone who has the url can obtain the video.

The idea needs one more trick to validate the download. I suppose one could compare a chunk of yt to the same piece downloaded over BitTorrent but perhaps there are better ideas to be had.

Eventually, with tit for tad one could swap one chunk of one video with a chunk from a different video on the same channel.

Re: Yark: Advanced and easy YouTube archiver now stable

#105

Post by bertman »

causality0 wrote: post #21

Does this have the ability to bet set to "grab highest available resolution" instead of specifying one? A lot of the material I'd like to archive has material from well before and after Youtube started supporting HD resolutions.

Looks like the "best" quality setting is hardcoded currently:

https://github.com/Owez/yark/blob/676074ee3d9e379d15e52ffe2e...

It would be nice to expose this setting via config file and/or the cli.

Re: Yark: Advanced and easy YouTube archiver now stable

#106

Post by 7h3b8duvwi »

seligman99 wrote: post #61
prometheus76 wrote: post #11

yt-dlp and a batch file that runs via Task Scheduler has been doing this for me for a couple of years now. I also grab the captions and throw that into a database so that I can search transcripts for a clip that I can remember but can't remember which video it's in. It was a fun weekend project.

Long ago I had my podcast downloader keep all files it downloads and recently I've been using OpenAI's Whisper to go through and create transcripts of the 8000 or so hours of data I have downloaded over the years. It's very cool to be able to search through and remind myself of something I heard once. Not exactly life changing, but still, nice to be able to quickly drill down and find audio for something when a curio…

8000 hours? Napkin math time, that's 20 years of 10+ hours daily.

I call BS.

Re: Yark: Advanced and easy YouTube archiver now stable

#108

Post by Minor49er »

mozman wrote: post #54
j1elo wrote: post #45

Afrer reading the description, this project seems to be solely focused on downloading all of a specific channel's videos. I've been taking my first steps at having a home server, and one of the things I'd love to do with it is having an archive of the videos that I have saved in my private playlists on YouTube. In my mind, the service would periodically check all my playlists, compare with what exists locally, and do…

I was doing this for awhile but it became expensive - tens of TBs on an expensive NAS just to hoard data.

Why not archive seldom-accessed files onto BD-ROMs? They're cheap to store

Re: Yark: Advanced and easy YouTube archiver now stable

#109

Post by defined »

7h3b8duvwi wrote: post #106
seligman99 wrote: post #61

Earlier quoted context omitted.

Long ago I had my podcast downloader keep all files it downloads and recently I've been using OpenAI's Whisper to go through and create transcripts of the 8000 or so hours of data I have downloaded over the years. It's very cool to be able to search through and remind myself of something I heard once. Not exactly life changing, but still, nice to be able to quickly drill down and find audio for something when a curio…

8000 hours? Napkin math time, that's 20 years of 10+ hours daily. I call BS.

My math says 365 days x 10 hours/day = 3650 hours. 8000 hours is just over 2 years, not 20.
Post reply on HN