Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thank you for helping us increase our bandwidth

blog.archive.org

Post reply on HN
Print view View original

131–140 of 207 posts

Re: Thank you for helping us increase our bandwidth

#131

Post by teekert »

I clicked this article to look for ways to contribute.. bandwidth. But there is no torrent/ipfs like method to help, just cash. I'd be happy to run a service that would help Wikipedia, archive.org, ddg (any service I believe in) decentralize. I always keep my Ubuntu torrents open for launch days and some beyond. I have a server running and I don't need my 50/50 fiber all the time.

If I was a better software dev, I'd try to make a daemon which I can feed a list of websites which I want to support with my caches that can be upgraded (like the DNS system?). Like a folding at home for bandwidth.

Re: Thank you for helping us increase our bandwidth

#132

Post by jl6 »

Mixed feelings here... I love the IA but I feel people should not use them as a streaming service. Their mission is preservation, and access is obviously part of that, but I would feel bad hammering them every day for general TV watching purposes.

Re: Thank you for helping us increase our bandwidth

#133

Post by lihaciudaniel »

Sometimes I wonder if people keep track of internet history. For example, we have the writing how Rome and Republic of Rome was only because of Cicero otherwise we wouldn't be able to comprehend. I think the modern day Herodotus is this website right here (assuming it will survive centuries hopefully)

Re: Thank you for helping us increase our bandwidth

#134

Post by TeMPOraL »

teekert wrote: post #131

I clicked this article to look for ways to contribute.. bandwidth. But there is no torrent/ipfs like method to help, just cash. I'd be happy to run a service that would help Wikipedia, archive.org, ddg (any service I believe in) decentralize. I always keep my Ubuntu torrents open for launch days and some beyond. I have a server running and I don't need my 50/50 fiber all the time. If I was a better software dev, I'd…

> just cash

This is almost always the best and most efficient way you can help a cause. You donating infrastructure means donating them extra work - they'll need to integrate and keep track of it, as well as manage a relationship with you (in particular, a risk of problems with you or your service). Meanwhile, with extra cash, they can buy what they need and what integrates well, or pay the most effective specialists they need.

It's an universal principle. Cash is the best gift, if you're gifting to help. That's why e.g. Red Cross frowns at people donating stuff - it creates a huge logistical problem for them as well as depriving them of opportunity to boost the markets in the disaster-struck area. Or why it's better for you to donate money to your local homeless shelter rather than volunteer to work in it - if you work your job for extra hours instead, and donate that money, they'll hire workers better skilled for that task.

Re: Thank you for helping us increase our bandwidth

#135

Post by PaywallBuster »

Someone1234 wrote: post #20

I donate to the Internet Archive (recurring). I just hope they stick to their core mission, and don't get sidetracked like Wikipedia has. They offer a valuable service and I believe will help historians in the future understand this point in the internet's life. PS - Don't forget to pick an Amazon Smile charity and use the Smile.Amazon sub-domain. Donated almost $20 to the EFF that way.

They already kind of have no?

They go from archiving the web (simple websites), itself a huge feat, to archiving a bunch of videos from justin.tv as long as they have 10 views?

There's more cases of websites shutting down, and AI feeling the need to archive everything.

Do we really need it?

Re: Thank you for helping us increase our bandwidth

#136

Post by bits_n_bytes »

jonah-archive wrote: post #115
cperciva wrote: post #40

Earlier quoted context omitted.

60 Gbps is about 1/3 of the traffic served by a single Netflix CDN node.

It's harder when you don't make money, and everyone's not downloading the same things.

I understand money is tight for a non-profit, but why are bandwidth costs a gating factor?

Both your upstreams, Cogent and Hurricane Electric, offer 100G ports at fiveish grand per month in carrier neutral DCs. Given that your budget is in the millions, an outlay of this magnitude doesn't seem wildly out of the question.

If you can explain what the problems are in getting more bandwidth, I'd be more than happy to see what I can do to help.

Please let me know if you'd prefer to discuss the matter over email.

Re: Thank you for helping us increase our bandwidth

#137

Post by TeMPOraL »

phonon wrote: post #100
EdJiang wrote: post #88

Earlier quoted context omitted.

You can get an 8TB HDD for $150 right now. That's 6250 drives. That's about $1MM in drives, which doesn't sound that cost-prohibitive. Obviously that's not the whole cost since you need to pay for bandwidth, replication and other infrastructure like the host node, but it sounds like something that could be even hosted by a number of volunteers on r/datahoarder or r/homelab. I also remember reading about Sia on HN, wh…

BackBlaze B2 is $5/TB/month Azure Archive is $2/TB/month ($1.68 if reserved) AWS Glacier Deep Archive is $1/TB/month GCP Cloud Storage Archive is $1.20/TB/month Of course, there can be i/o and network charges, and different levels of redundancy (but possibly bulk discounts)...but the bare storage costs for for 50 PB per year would be roughly $600k - $3 MM/y.

Aren't the costs of getting that data out of the backup much larger than the cost of keeping it in the first place, to the point that when you actually need to restore a large backup, it turns out it was better to have been managing it yourself? That's the impression I got from various HN comments on the topic over the years.

Re: Thank you for helping us increase our bandwidth

#138

Post by robocat »

dmd wrote: post #61
mostlysimilar wrote: post #2

archive.org feels like an irreplaceable treasure, the Wayback Machine alone is a time capsule of our digital history. I donate to them monthly and know a lot of other people do as well, so I don't worry much about their financial stability. I'm more worried about external pressures taking content down. I hope the data is backed up six ways to sunday, and that somewhere there's a plan to make it all accessible if Inte…

I'm very worried about its backups and what happens when the next Big One hits SF. As far as I've ever been able to determine from talking to anyone at IA (e.g., Kahle, Scott), they don't really have any sort of backups that could actually be restored from in a disaster situation.

Do they have distributed backup?

Surely a lot of people would be happy to donate unused space on their drives (I know I would) - especially since backup wouldn’t use much bandwidth...

Re: Thank you for helping us increase our bandwidth

#139

Post by TeMPOraL »

EdJiang wrote: post #88
dmd wrote: post #71

Earlier quoted context omitted.

The IA is about 50 PB. IABAK stores 100 TB, or 0.2% of it.

You can get an 8TB HDD for $150 right now. That's 6250 drives. That's about $1MM in drives, which doesn't sound that cost-prohibitive. Obviously that's not the whole cost since you need to pay for bandwidth, replication and other infrastructure like the host node, but it sounds like something that could be even hosted by a number of volunteers on r/datahoarder or r/homelab. I also remember reading about Sia on HN, wh…

I think at this scale we're no longer talking about buying drives, but securing a steady supply stream of them. So we're talking not $1MM, but a somewhat safe $1MM per year for it to be even worth considering.

Re: Thank you for helping us increase our bandwidth

#140

Post by TeMPOraL »

ramraj07 wrote: post #90
Endlessly wrote: post #85

Earlier quoted context omitted.

Since numbers are not 100% clear... (50 petabytes * 0.2% = 100 terabytes) [$0.005 ($/GB/Month) BackBlaze cost] [(50 petabytes) / (1 gigabyte) = 50,000,000] (50,000,000 * $0.005 = $250,000 US$) ————— Meaning based on my numbers, that is $250,000 USD a month to host 50 petabytes of data on BackBlaze.

That's a fraction of the AWS bills for many startups arguably doing absolutely nothing

They have VC money to burn. Archive.org doesn't.

Also, those backups would be (relatively) cheap to keep, but not necessarily to restore.

Post reply on HN