Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YouTube as Storage

github.com

Post reply on HN
Print view View original

61–70 of 163 posts

Re: YouTube as Storage

#61

Post by ntoskrnl_exe »

repeekad wrote: post #4

I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches” They said it didn’t matter, because the sheer volume of new data flowing in growing so fast made the old data just a drop in the bucket

Wouldn't it also be a performance nightmare?

The energy bill for scanning through the terabytes of metadata would be comparable to that of several months of AI training, not to mention the time it would take. Then deleting a few million random 360p videos and putting MrBeast in their place would result in insane fragmentation of the new files.

It might really just be cheaper to keep buying new HDDs.

Re: YouTube as Storage

#62

Post by moffkalast »

ntoskrnl_exe wrote: post #61
repeekad wrote: post #4

I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches” They said it didn’t matter, because the sheer volume of new data flowing in growing so fast made the old data just a drop in the bucket

Wouldn't it also be a performance nightmare? The energy bill for scanning through the terabytes of metadata would be comparable to that of several months of AI training, not to mention the time it would take. Then deleting a few million random 360p videos and putting MrBeast in their place would result in insane fragmentation of the new files. It might really just be cheaper to keep buying new HDDs.

Besides with their search deteriorating to the point where a direct video title doesn't result in a match, nobody can see those videos anyway and they don't have to cache them.

Re: YouTube as Storage

#63

Post by dev1ycan »

ntoskrnl_exe wrote: post #61
repeekad wrote: post #4

I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches” They said it didn’t matter, because the sheer volume of new data flowing in growing so fast made the old data just a drop in the bucket

Wouldn't it also be a performance nightmare? The energy bill for scanning through the terabytes of metadata would be comparable to that of several months of AI training, not to mention the time it would take. Then deleting a few million random 360p videos and putting MrBeast in their place would result in insane fragmentation of the new files. It might really just be cheaper to keep buying new HDDs.

This is why they removed searching for older videos (specific time) and why their search pushes certain algorithmic videos, other older videos when found by direct link are on long term storage and take a while to start loading.

Re: YouTube as Storage

#64

Post by stogot »

ntoskrnl_exe wrote: post #61
repeekad wrote: post #4

I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches” They said it didn’t matter, because the sheer volume of new data flowing in growing so fast made the old data just a drop in the bucket

Wouldn't it also be a performance nightmare? The energy bill for scanning through the terabytes of metadata would be comparable to that of several months of AI training, not to mention the time it would take. Then deleting a few million random 360p videos and putting MrBeast in their place would result in insane fragmentation of the new files. It might really just be cheaper to keep buying new HDDs.

S3 allows delete and is efficient here. I’m sure Google can figure it out

They allow search by timestamp, I’m sure YouTube can write algo to find zero <=1 view

Re: YouTube as Storage

#65

Post by komali2 »

agnishom wrote: post #47
komali2 wrote: post #30

Earlier quoted context omitted.

Yes, but it's a classic story of what actually happened to the commons - they were fenced and sold to land "owners." Honestly, if you aren't taking full advantage within the constraints of the law of workarounds like this, you're basically losing money. Like not spending your entire per diem budget when on a business trip.

This seems like a narrow understanding of value. Which do you think has more value to me? (a) I save some money by exploiting the storage loophole (b) The existence of a cultural repository of cat videos, animated mathematics explainers, long video essays continue to be available to (some parts of) humanity (for the near future).

This is assuming doing A has any meaningful impact on B.

Anyway in this situation it's less that YouTube is providing us a service and more, it's captured a treasure trove of our cultural output and sold it back to us. Siphoning back as much value as we can is ethical. If YouTube goes away, we'll replace it - PeerTube or other federated options are viable. The loss of the corpus of videos would be sad but not catastrophic - some of it is backed up. I have ~5Tb of YouTube backed up, most of it smaller channels.

I agree generally with you that the word "value" is overencompassing to the point of absurdity though. Instrumental value is equated with moral worth, personal attachment, and distribution of scarcity. Too many concepts for one word.

Re: YouTube as Storage

#66

Post by Gigachad »

Kwpolska wrote: post #28
arjie wrote: post #22

Earlier quoted context omitted.

Videos do disappear, though. https://www.reddit.com/r/DataHoarder/comments/1ioz4x1/is_it_... Searching hn.algolia.com for examples will yield numerous ones. https://news.ycombinator.com/item?id=23758547 https://bsky.app/profile/sinevibes.bsky.social/post/3lhazuyn...

Of course videos disappear for copyright, ToS violations, or when the uploaders remove them. They do not disappear just because nobody watched them.

There’s a whole activity around discovering random 15 year old videos with almost no views. It’s usually some random home video

Re: YouTube as Storage

#67

Post by jl6 »

repeekad wrote: post #4

I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches” They said it didn’t matter, because the sheer volume of new data flowing in growing so fast made the old data just a drop in the bucket

One day, it will matter. Not even Google can escape the consequences of infinite growth. Kryder's Law is over. We cannot rely on storage getting cheaper faster than we can fill it, and orgs cannot rely on being able to extract more value from data than it costs to store it. Every other org knows this already. The only difference with Google is that they have used their ad cash generator to postpone their reality check moment.

One day, somebody is going to be tasked with deciding what gets deleted. It won't be pretty. Old and unloved video will fade into JPEG noise as the compression ratio gets progressively cranked, until all that remains is a textual prompt designed to feed an AI model that can regenerate a facsimile of the original.

Re: YouTube as Storage

#68

Post by sfn42 »

moffkalast wrote: post #62
ntoskrnl_exe wrote: post #61

Earlier quoted context omitted.

Wouldn't it also be a performance nightmare? The energy bill for scanning through the terabytes of metadata would be comparable to that of several months of AI training, not to mention the time it would take. Then deleting a few million random 360p videos and putting MrBeast in their place would result in insane fragmentation of the new files. It might really just be cheaper to keep buying new HDDs.

Besides with their search deteriorating to the point where a direct video title doesn't result in a match, nobody can see those videos anyway and they don't have to cache them.

It's not just the search deteriorating. The frontend is littered with bugs. If you write a comment and try to highlight and delete part of that comment, it'll often delete the part you didn't highlight. So apparently they implemented their own textfield for some reason and also fucked it up. It's been like that for years.

The youtube shorts thing is buggy as shit, it'll just stop working a lot of the time, just won't load a video. Some times you have to go back and forth a few times to get it to load. It'll often desync the comments from the video, so you're seeing comments from a different video. Some times the sound from one short plays over the visuals of another.

It only checks for notifications when you open the website from a new tab, so if you want to see if you have any notifications you have to open youtube in a new tab. Refreshing doesn't work.

Seems like all the competent developers have left.

Re: YouTube as Storage

#70

Post by rzzzt »

iberator wrote: post #50
rzzzt wrote: post #44

Earlier quoted context omitted.

Upload to other video sharing sites for redundancy. RAIVS!

Stop ruining the internet end exploiting free resources

It was a tongue-in-cheek / silly suggestion outright. I don't think many people are actually using the tool for its off-ToS purpose though, there is also a lot of prior art across multiple sharing services. It's still interesting to think about the inner workings of it.
Post reply on HN