Yark: Advanced and easy YouTube archiver now stable • Hacker News Board

Re: Yark: Advanced and easy YouTube archiver now stable

#91

Post by gcau » Fri, Jan 06, 2023, 4:15 AM UTC

masukomi wrote: post #24

okasaki wrote: post #22

Earlier quoted context omitted.

But most people have more free disk space than 10gb, and most people also have more than 1 device. I have 2 phones, 3 laptops, a desktop and a NAS, with some 40TB between them. My workplace has a private cloud with some 70PB of storage, plus tape, and tons of desktops and laptops.

a) i seriously doubt your "most" claim about free disk space. Maybe "most privileged white folks in rich countries" but not "most people" b) just because i HAVE 10Gb of free disk space doesn't mean i'm going to offer it up for archiving of random internet crap c) if it was on my phone it'd cost money and now we're very actively ignoring just how expensive it is to be online in 3rd world countries if you're not a rich…

>privileged white folks in rich countries

Why bring race into it at all? Do privileged black/asian/etc people not exist? I'm sure you didn't mean it, but it's a very shortsighted thing to say.

Also, I think you're taking OPs comment too literally. Yes, not everyone has several devices and not everyone would want to store random videos on their devices, the point being though that 10gb is a veryy conservatively small number - and it being a fun thought experiment to imagine things like this.

Re: Yark: Advanced and easy YouTube archiver now stable

#92

Post by midoridensha » Fri, Jan 06, 2023, 4:18 AM UTC

rchaud wrote: post #41

CamelCaseName wrote: post #18

Earlier quoted context omitted.

It's unbelievable that YouTube was as free as it was for as long as it was. We got too great a deal for so long that we many people can't see things any other way.

It's unbelievable that Wikipedia is free and survives on donations. Youtube sells ads and is subsidized by one of the biggest ad companies in the world that happens to have a lot of cheap cloud storage available.

My understanding is that Wikipedia does not survive on, or even need donations, at all: they are entirely funded through their endowment.

The donations go to the Wikimedia Foundation which spends the money on a bunch of other social stuff that's not related to running Wikipedia at all. So unless you want to support all those other causes, you're effectively wasting your money by donating to Wikipedia's frequent donation requests. Wikipedia isn't the storage hog that YouTube is; it doesn't cost that much to keep it afloat, and the people moderating the content (the editors) are all volunteers anyway.

Re: Yark: Advanced and easy YouTube archiver now stable

#93

Post by iKlsR » Fri, Jan 06, 2023, 4:23 AM UTC

This is pretty cool, I did a similar personal project that I describe here https://news.ycombinator.com/item?id=28480790. The only historical thing I log however is if a video was removed/reuploaded.

Re: Yark: Advanced and easy YouTube archiver now stable

#94

Post by pvg » Fri, Jan 06, 2023, 5:04 AM UTC

wahnfrieden wrote: post #88

pvg wrote: post #70

Earlier quoted context omitted.

There have been iOS video players with such functionality built-in whose authors have had to remove the functionality at Apple's request.

Thank you. I have a couple like that and didn’t know how confident to be with replicating It seems Apple is fine with adblocking by default for web content, but inconsistently as with youtube. Hard to predict what's risky to invest dev time into

I'm not an expert on Apple's rules but my understanding is the download thing is circumvention of the terms under which Youtube licenses/re-licenses content so it's treated fairly strictly.

Re: Yark: Advanced and easy YouTube archiver now stable

#95

Post by alwayslikethis » Fri, Jan 06, 2023, 5:08 AM UTC

wahnfrieden wrote: post #88

pvg wrote: post #70

Earlier quoted context omitted.

There have been iOS video players with such functionality built-in whose authors have had to remove the functionality at Apple's request.

Thank you. I have a couple like that and didn’t know how confident to be with replicating It seems Apple is fine with adblocking by default for web content, but inconsistently as with youtube. Hard to predict what's risky to invest dev time into

Apparently they don't like when your app does something in a way that breaks ToS or EULA, even if they are not worth the pixels they are written on. Apple being the trillion dollar company it is, is naturally averse to the legal risk posed by having a ToS-breaking app on their store. This is why having a single company controlling your entire device is bad -- they do what's best for them, rather than whatever you might happen to want.

Re: Yark: Advanced and easy YouTube archiver now stable

#96

Post by alwayslikethis » Fri, Jan 06, 2023, 5:11 AM UTC

HPsquared wrote: post #60

mozman wrote: post #54

Earlier quoted context omitted.

I was doing this for awhile but it became expensive - tens of TBs on an expensive NAS just to hoard data.

You can probably set flags to download in lower bitrate formats. Some formats use a LOT less data than others, and usually the extra quality isn't really needed anyway if it's only being kept for reference. The difference between 1080p (or, say, 4K60) and 240p or 360p is massive.

Also the codec matters. AV1 is quite a bit smaller than VP9, which is in turn smaller than x264. Of course the downside is AV1 is harder to decode. For the purpose of well-encoded content, you can estimate that the bitrate doubles whenever you double both dimensions of the video, e.g. 720p would be twice the bitrate of 360p and half of 1440p.

Re: Yark: Advanced and easy YouTube archiver now stable

#97

Post by ekianjo » Fri, Jan 06, 2023, 5:25 AM UTC

Owez wrote: post #2

I've been working on polishing my YouTube archiver project for the last while and I've finally released a solid version of it, it has an offline web viewer/visualiser for archived channels and it's managed using a cli. Most importantly, its easy to use :)

Is there a longer documentation anywhere? It's not clear from the README if you add a whole channel when you create an archive, or if you can add videos one by one to archive them?

Re: Yark: Advanced and easy YouTube archiver now stable

#98

Post by ekianjo » Fri, Jan 06, 2023, 5:29 AM UTC

rchaud wrote: post #41

CamelCaseName wrote: post #18

Earlier quoted context omitted.

It's unbelievable that YouTube was as free as it was for as long as it was. We got too great a deal for so long that we many people can't see things any other way.

It's unbelievable that Wikipedia is free and survives on donations. Youtube sells ads and is subsidized by one of the biggest ad companies in the world that happens to have a lot of cheap cloud storage available.

> It's unbelievable that Wikipedia is free and survives on donations.

Wikipedia is mostly text. You can download the whole thing and fit it on a SD card.

Re: Yark: Advanced and easy YouTube archiver now stable

#99

Post by googlryas » Fri, Jan 06, 2023, 6:20 AM UTC

Owez wrote: post #2

I've been working on polishing my YouTube archiver project for the last while and I've finally released a solid version of it, it has an offline web viewer/visualiser for archived channels and it's managed using a cli. Most importantly, its easy to use :)

Great project! I have a playlist that I use to keep track of videos that my young kid likes to watch, but wanted to get away from yt because of the ads, related videos, comments, autoplaying next vids, etc. So I just followed the simple instructions and bam, I have all the videos on my computer now with a UI that is even easier to use for him.

Very nice, thank you!

Re: Yark: Advanced and easy YouTube archiver now stable

#100

Post by ebb_earl_co » Fri, Jan 06, 2023, 7:16 AM UTC

seligman99 wrote: post #61

prometheus76 wrote: post #11

yt-dlp and a batch file that runs via Task Scheduler has been doing this for me for a couple of years now. I also grab the captions and throw that into a database so that I can search transcripts for a clip that I can remember but can't remember which video it's in. It was a fun weekend project.

Long ago I had my podcast downloader keep all files it downloads and recently I've been using OpenAI's Whisper to go through and create transcripts of the 8000 or so hours of data I have downloaded over the years. It's very cool to be able to search through and remind myself of something I heard once. Not exactly life changing, but still, nice to be able to quickly drill down and find audio for something when a curio…

What kind of hardware do you have that makes it feasible to process thousands of hours of podcasts? I want to do the same but I’ve heard that Whisper requires some serious GPU might for decent accuracy (Linux Unplugged podcast specifically).