Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keep Our Servers Running

blog.archive.org

Post reply on HN
Print view View original

211–220 of 292 posts

Re: Keep Our Servers Running

#211

Post by jcrawfordor »

arjie wrote: post #10

I do like the Internet Archive but I wish I could retrieve large segments from it myself. It has very aggressive 429s. It would be a nice higher donor tier feature. Maybe it’s not possible to do so and retain 501c3 status? Regardless I have had to start performing personal datahoarding. C’est la vie.

Yesterday I discovered that WayBackMachine appears to be serving 429s to all Chrome-based browsers, across all network connections. Using Firefox (or a firefox UA) fixes it. I assume this is a temporary situation, or even by mistake, but underscores that archive.org is getting creaky and hard to actually use.

Re: Keep Our Servers Running

#212

Post by [deleted] »

tossandthrow wrote: post #109

2:1 by who? That is not written. These types of schemes ring very poorly in my head for charity. If someone has the ability to give 2$ for every 1$ I give, why are they only giving on contingence? Just give the money!

[deleted]

Re: Keep Our Servers Running

#213

Post by justin66 »

ndriscoll wrote: post #191
justin66 wrote: post #182

Earlier quoted context omitted.

> a library worker doesn't need their computer at 9am so badly that they need an on-call IT worker (and presumably some kind of middle-of-the-night monitoring) to ensure it before they get in. A single worker probably doesn't, yes. In contrast, hundreds of people being unable to do anything when they come to work in the morning is not a terrible reason to call someone and have them login for a few minutes. Back to th…

If it's only a few minutes, you could just as well fix it at the start of the day. And if it's not, oh well? People can still read books. The librarian can still help them to find a book. If they want to manually track until the computer is back up, they could even still do checkouts. If those users just say "oh, okay, I'll check back in a couple hours", then okay. Again, this is not hundreds of thousands of people w…

At this point I think you’re just committed to the bit but you cannot possibly believe what you’re writing, as it’s a blinkered analysis and truly misanthropic in its effect. A metropolitan library system employs hundreds of people in a variety of roles and if we can prevent them from having a frustrating and wasteful start to their day, it’s worth an on-call person’s time.

> and on-call has a human cost.

In the context of this thread, we’re talking about this job description:

Senior Datacenter Network Infrastructure Engineer Full Time San Francisco, CA, US 30+ days ago Requisition ID: XXXX Salary Range: $140,000.00 To $200,000.00 Annually

It’s not like we’re talking about indentured servitude or something.

> Perhaps put another way, is there any computer service now that doesn't need a guarantee? Or is everything just critical now?

That’s an interesting strawman you have there. I thought I killed it upthread when I wrote “Depends on the service.”

Re: Keep Our Servers Running

#215

Post by toomuchtodo »

riobard wrote: post #177
prirun wrote: post #142

I read that Internet Archive keeps 2 copies of all their data. Storj, the decentralized storage service, is currently in Chap 11 bankruptcy, mostly IMO because of incompetent management. They pay their node operators, the guys that actually maintain the hard drive space, $1.35/TB/mo to store data. (However it costs Storj 1.8x that because of erasure coding). They still have way more available capacity going unused, s…

Maybe they could start a Archive@Home project for volunteers to donate unused disk space for the greater good.

https://getdweb.net/ is that project.

https://news.ycombinator.com/item?id=29639222

> As noted the Internet Archive is experimenting with filecoin.io and storj.io and is always open to suggestions about how we might do our jobs better, and improve our service. We also host regular meetups (and have hosted summits and a camp) related to the Decentralized Web. See: https://blog.archive.org/tag/dweb/

Inside The Internet Archive's Infrastructure - https://news.ycombinator.com/item?id=46613324 - February 2026 (119 comments)

u/stavros proposed "Elephant" as well, which could be implemented today without Archive.org's explicit assistance.

Elephant system design - https://gist.github.com/skorokithakis/68984ef699437c5129660d... (A distributed, voluntary backup system (high-level design document))

https://news.ycombinator.com/item?id=46637992 (additional context)

(TLDR Every Archive.org item has a torrent, enabling distributed replication and serving of archived items and their contents, Wayback is a bit trickier; you can also ask your local nation state and museum/preservation apparatus to colocate some IA racks, if one is so inclined; The Internet Archive’s entire annual budget ($30M-$40M) covers 200PB storage plus all operations)

Re: Keep Our Servers Running

#216

Post by toomuchtodo »

prirun wrote: post #142

I read that Internet Archive keeps 2 copies of all their data. Storj, the decentralized storage service, is currently in Chap 11 bankruptcy, mostly IMO because of incompetent management. They pay their node operators, the guys that actually maintain the hard drive space, $1.35/TB/mo to store data. (However it costs Storj 1.8x that because of erasure coding). They still have way more available capacity going unused, s…

The Internet Archive can store data in perpetuity for ~$3/GB one time (estimated based on recent AI driven hardware inflation, used to be $2/GB).

https://blog.dshr.org/2026/01/internet-archives-storage.html

https://blog.dshr.org/2021/03/internet-archive-storage.html

https://archive-it.org/vault/

Re: Keep Our Servers Running

#217

Post by badlibrarian »

smolder wrote: post #144
abeyer wrote: post #38

"Keep running" is interesting phrasing, given that wayback has been serving me nothing but some mix of timeouts, 429 errors, and "Temporarily Offline" messages for weeks.

I've watched whole video series from their archives. What are you doing to get throttled?

Which archives? Wayback is different from Archive. Which is different from Archive-It. And the problem with Wayback was power poles being set on fire by homeless people, then SF power in general, then "upgrades," then "bad actors," then hot weather causing the machines to slow down because they don't have air conditioning, evil AI companies, and now seems to be the fact that an organization backed by someone with nearly a billion dollars, founded 30 years ago, cannot source hard drives.

https://bsky.app/profile/textfiles.com/post/3musdwx6kqk2c

Fundraising drive coordinated with their loudest employee saying something else, because... of course.

Re: Keep Our Servers Running

#218

Post by LastTrain »

crewman9 wrote: post #28
LastTrain wrote: post #23

Earlier quoted context omitted.

They are independent organizations with separate funding.

then why did the main IA account post about this https://mastodon.archive.org/@internetarchive/11721950124572... and it's darkly funny how HN downvotes this when every single answer on Mastodon is incredibly negative

I swear sometimes people get so offended by something they stop thinking. It doesn't have to be all or nothing, you know? Like, you can acknowledge that these are two separate and independent organizations while also having the opinion that you don't like what Internet Archive Europe is doing in this one case and that you feel like IA posting about it is a tacit endorsement of the thing you don't like. Maybe it is enough for you to not support them, that's fine, but don't be intellectually dishonest about it.

Re: Keep Our Servers Running

#219

Post by ndriscoll »

justin66 wrote: post #213
ndriscoll wrote: post #191

Earlier quoted context omitted.

If it's only a few minutes, you could just as well fix it at the start of the day. And if it's not, oh well? People can still read books. The librarian can still help them to find a book. If they want to manually track until the computer is back up, they could even still do checkouts. If those users just say "oh, okay, I'll check back in a couple hours", then okay. Again, this is not hundreds of thousands of people w…

At this point I think you’re just committed to the bit but you cannot possibly believe what you’re writing, as it’s a blinkered analysis and truly misanthropic in its effect. A metropolitan library system employs hundreds of people in a variety of roles and if we can prevent them from having a frustrating and wasteful start to their day, it’s worth an on-call person’s time. > and on-call has a human cost. In the cont…

No, I quite strongly believe it actually. I likewise find it hard to believe you're serious, so I'll raise my question again: if not even a municipal library (which, I'll remind you, aren't even open the majority of the time and we all get by just fine), what is an example of a non-critical service in your mind? Maybe the online checkin for a specific Great Clips? It depends on what? Pointing out that "it depends on the service", and that a library clearly isn't critical was my whole original point.

Misanthropic to me is making every IT job require getting up in the middle of the night to save a 100*epsilon of theoretical frustration lest we be perceived as lacking the requisite seriousness. And when someone points out that actually a given case is really not a big deal, we're met with "but hundreds of thousands of people might think 'oh, darn' for a moment!" And for a library, IME librarians tend to be really chill, so I suspect they'd, like me, attest that actually they'd prefer you not work at midnight on their account. Like I said, as a citizen funding and using my library, I sure hope they don't treat their employees that way.

Re: Keep Our Servers Running

#220

Post by phendrenad2 »

Why would I donate money to keep your servers running, when I get my IP blocked for days when I try to view more than 20 archived pages? What utility are you actually providing me? From my perspective, Internet Archive is already defunct. It no longer exists, just a hollow organization pretending to be Internet Archive remains.
Post reply on HN