Live data from Hacker News

YouTube as Storage

github.com

91–100 of 163 posts

Re: YouTube as Storage

#91

Earlier quoted context omitted.

The problem is that some content creators have already passed away (and others will pass away by then), and their videos will likely be deleted forever.

Monuments erode away and memories of those enshrined are lost time as well, nothing lasts forever.

    I met a user from an antique land
    Who said: Two squares of a clip of video
    Stand in at the end of the search. Near them,
    Lossly compressed, a profile with a pfp, whose smile,
    And vacant eyes, and shock of content baiting,
    Tell that its creator well those passions read
    Which yet survive, stamped on these unclicked things,
    The hand that mocked them and the heart that fed:
    And on the title these words appear:
    "My name is Ozymandias, Top Youtuber of All Time:
    Look on my works, ye Mighty, and like and subscribe!"
    No other video beside remains. Round the decay
    Of that empty profile, boundless and bare
    The lone and level page stretch far away.

Re: YouTube as Storage

#93

I don't get how it works. > Encoding: Files are chunked, encoded with fountain codes, and embedded into video frames Wouldn't YouTube just compress/re-encode your video and ruin your data (assuming you want bit-by-bit accurate recovery)? If you have some redundancy to counter this, wouldn't it be super inefficient? (Admittedly, I've never heard of "fountain codes", which is probably crucial to understanding how it wo…

Yeah, I would assume that transcodes kill this eventually...

Re: YouTube as Storage

#94

Earlier quoted context omitted.

The problem is that some content creators have already passed away (and others will pass away by then), and their videos will likely be deleted forever.

That may be, but I assume for videos that had some viewership base, there may be a consideration. E. g. if a video was viewed 20 million times, it may be worth more than one that was viewed only 5 times.

I've stumbled upon very valuable content with very low view numbers - the algorithms spiral around spectacularity and provocation, not quality or insight.

Re: YouTube as Storage

#95
post #14

Earlier quoted context omitted.

Just make sure you have you have a bot network storing the information in with multiple accounts. Also with with enough parity bits (E.g. PAR2) to recover broken vids or removed accounts.

par2 is very limited. It only support 32k parts in total (or in reality that means in practice 16k parts of source and 16k parts of parity). Lets take 100GB of data (relatively large, but within realm of reason of what someone might want to protect), that means each part will be ~6MB in size. But you're thinking you also created 100GB of parity data (6MB*16384 parity parts) so you're well protected. You're wrong. Now…

you can split files so you can have more par blocks (100GB in 100 1GB parts 32k blocks per part)

Re: YouTube as Storage

#96
post #4

I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches” They said it didn’t matter, because the sheer volume of new data flowing in growing so fast made the old data just a drop in the bucket

Now that they can harvest it all for AI training, that decision was the cheapest and greatest thing they ever did.

Imagine trying to pay for all that content, nobody on earth would be able or willing to supply it.

Re: YouTube as Storage

#97

Earlier quoted context omitted.

The problem is that some content creators have already passed away (and others will pass away by then), and their videos will likely be deleted forever.

Monuments erode away and memories of those enshrined are lost time as well, nothing lasts forever.

Like tears in rain <3

Re: YouTube as Storage

#98

Earlier quoted context omitted.

Wouldn't it also be a performance nightmare? The energy bill for scanning through the terabytes of metadata would be comparable to that of several months of AI training, not to mention the time it would take. Then deleting a few million random 360p videos and putting MrBeast in their place would result in insane fragmentation of the new files. It might really just be cheaper to keep buying new HDDs.

This is why they removed searching for older videos (specific time) and why their search pushes certain algorithmic videos, other older videos when found by direct link are on long term storage and take a while to start loading.

Well the time filters (before/after:date) still seem to work, but for controversial / hot topics, somehow, more recent videos tend to still show up at the top. Try "scandal after:2010 before:2012"..

Re: YouTube as Storage

#99
post #67

Earlier quoted context omitted.

One day, it will matter. Not even Google can escape the consequences of infinite growth. Kryder's Law is over. We cannot rely on storage getting cheaper faster than we can fill it, and orgs cannot rely on being able to extract more value from data than it costs to store it. Every other org knows this already. The only difference with Google is that they have used their ad cash generator to postpone their reality chec…

It depends. At the rough 2 PB of new data they get a day that’s about 10 sq ft of physical rack space per day. Each data center is like 500,000 sq feet so each data center can hold 120 years of YouTube uploads. They’re not going to have to restrict uploads anytime soon.

Not all of the square footage of a data center is usable for racks

Re: YouTube as Storage

#100

Earlier quoted context omitted.

The problem is that some content creators have already passed away (and others will pass away by then), and their videos will likely be deleted forever.

Monuments erode away and memories of those enshrined are lost time as well, nothing lasts forever.

let's see what will last longer over the ages : engraved stone or google?
Post reply on HN