Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Internet Archive as a default host-of-record for startups

twitter.com

Post reply on HN
Print view View original

81–90 of 188 posts

Re: Internet Archive as a default host-of-record for startups

#81

Post by Ericson2314 »

luckylion wrote: post #72
Ericson2314 wrote: post #20

Earlier quoted context omitted.

There are clearly some things we, and the author, wants to persist, and yet the internet fails to do so. Trying to muddle in privacy concerns with the accumulation of public knowledge undermines the whole concept of a shared society, or there being even the potential of accumulating "progress" in the first place. Also, historically speaking, people with means used to save their letters for posterity, which proved to…

"You shouldn't have privacy because it undermines the concept of a shared society, and also future historians might find your life interesting." Thanks.

Rediculus Strawman

Re: Internet Archive as a default host-of-record for startups

#83

Post by mwattsun »

bananamerica wrote: post #68
mwattsun wrote: post #30

I think it's interesting to think about what we have lost because we couldn't keep everything from a 100 years ago and what society 100 years from now will be grateful we preserved. Off the top of my head, we lost a lot of common wisdom in dealing with the flu pandemic of 1918 because personal letters and most newspapers were not preserved. I think 100 years from now they might wish we had preserved more from margina…

Hard to know what will be of interest for future historians. Some things in which we place great value can be considered irrelevant, while some of our junk can become historical gold.

> some of our junk can become historical gold

That's what interests me. For example, there's a cool repository of 12 step speaker meeting talks hosted in Iceland [1] and frankly, some of the talks are junk, but there's a lot of wisdom. What I find interesting is how it showcases how ordinary citizens talk to each other. The words they use, the accents, the little gems of folk wisdom contained, along with some uncommon stories.

This will be valuable 100 years from now if, for example, you want to build a virtual world based in the mid to late 20th century and you want to get the accents correct. What phrases did people use? What were some common misconceptions? Maybe 100 years from now addiction will no longer be a problem. If my virtual world is to be accurate I need to know what it was like for ordinary people when it was a problem. Etc...

[1] https://xa-speakers.org/

Re: Internet Archive as a default host-of-record for startups

#84

Post by EGreg »

frakkingcylons wrote: post #50
EGreg wrote: post #12

Earlier quoted context omitted.

Why not use MaidSAFE or IPFS for that?

With regards to IPFS, you'd need to find a pinning service that offers long-term contracts. Moreover, I have more confidence in the Internet Archive in being around in 10, 20, 30 years compared to any existing pinning service. That's not meant to be a slight against said pinning services, just that the IA is well established in their role.

Check out Freenet

They drop stuff that has the least amount of accessing, so if you want to keep something online you have to pay someone to keep accessing it. Makes sense, but I'd argue that it should be a market, meaning the price should go up as more people try to access stuff, but anyone can start to seed popular content and collect revenues also for hosting it (this is better than wasting electricity on accessing stuff or doing proof of work).

But isn't FileCoin exactly that for IPFS?

MaidSAFE goes a step further and has nodes rebalance autonomously and earn the most safecoin, as something gets more popular it gets seeded more.

Re: Internet Archive as a default host-of-record for startups

#85

Post by toomuchtodo »

rmbyrro wrote: post #80
toomuchtodo wrote: post #39

Earlier quoted context omitted.

It costs the Internet Archive $2/GB to host content in perpetuity. They have a tool, Archive It, that will periodically crawl your site for archival purposes if you are not technical. For my needs, I run a report monthly for the content I’ve archived using my IA account to determine archived GBs, and then donate the amount needed to cover those costs. Consider reaching out to their patron services email address with…

$2/GB in perpetuity is really cheap. They should write a paper about how they did it, if they haven't already. Edit: I'm assuming they can deliver reliability and durability similar to modern cloud standards, like AWS S3.

Self built storage nodes and software for storage system, with an expectation that storage costs continue to decline per GB into the future.

Re: Internet Archive as a default host-of-record for startups

#86

Post by gojomo »

adventured wrote: post #76
simonw wrote: post #9

The feature I most want from the Internet Archive is the ability to donate them an old domain name and enough cash to renew it for the next hundred years such that they can keep an archived version of a site available (without breaking any incoming links) for a very long time. They would also need to be able to handle legal administration costs of things like DMCA take-down notices, but I assume they already have to…

> The feature I most want from the Internet Archive The feature I most want from IA is a streamlined system to delete content they have archived on domains that I own, including a proper privacy law compliance effort on their part. They have intentionally made it a difficult, manual process to get content removed. They operate as a de facto malicious crawler. They massively violate GDPR with how they operate and few…

Shouldn't it be hard to delete things from a library and historical-archive?

If you had to choose between the GDPR, & an accurate historical record, which would you prefer?

Re: Internet Archive as a default host-of-record for startups

#87

Post by polote »

The idea is more interesting when you think about scrapers. Take any ecommerce website, there are several scrapers that download all pages every hours, it would be more efficient if a provider had a live copy of the website and then serve the requests to the scrapers or could even send webhook.

A website could handle tons of scrappers without having high bandwidth, only the provider will need high bandwidth.

The issue is that scrapers often play with cookies and dynamic websites and such solution wouldn't work on these cases

Re: Internet Archive as a default host-of-record for startups

#88

Post by akouri »

polote wrote: post #87

The idea is more interesting when you think about scrapers. Take any ecommerce website, there are several scrapers that download all pages every hours, it would be more efficient if a provider had a live copy of the website and then serve the requests to the scrapers or could even send webhook. A website could handle tons of scrappers without having high bandwidth, only the provider will need high bandwidth. The issu…

scraper*

Re: Internet Archive as a default host-of-record for startups

#89

Post by Buttons840 »

toomuchtodo wrote: post #39
simonw wrote: post #9

The feature I most want from the Internet Archive is the ability to donate them an old domain name and enough cash to renew it for the next hundred years such that they can keep an archived version of a site available (without breaking any incoming links) for a very long time. They would also need to be able to handle legal administration costs of things like DMCA take-down notices, but I assume they already have to…

It costs the Internet Archive $2/GB to host content in perpetuity. They have a tool, Archive It, that will periodically crawl your site for archival purposes if you are not technical. For my needs, I run a report monthly for the content I’ve archived using my IA account to determine archived GBs, and then donate the amount needed to cover those costs. Consider reaching out to their patron services email address with…

I'd pay twice that to host my static blog with IA. ;)

I guess archiving a static blog is already trivial for their system, but I'd pay $100 a year to have IA host my static blog. The overhead I would consider as a donation to a worthy cause.

Then again, I can just give them $100 a year and find some free static hosting, like GitHub Pages, and call it a day.

Re: Internet Archive as a default host-of-record for startups

#90

Post by kaashif »

tablespoon wrote: post #62
toomuchtodo wrote: post #39

Earlier quoted context omitted.

It costs the Internet Archive $2/GB to host content in perpetuity. They have a tool, Archive It, that will periodically crawl your site for archival purposes if you are not technical. For my needs, I run a report monthly for the content I’ve archived using my IA account to determine archived GBs, and then donate the amount needed to cover those costs. Consider reaching out to their patron services email address with…

> It costs the Internet Archive $2/GB to host content in perpetuity. Do you have source/more info than that? Lets say the internet archive is 100 PB [1], that's 100,000,000 GB [2], and at that rate it comes out to $200 million [3] for the whole thing forever. That's a lot of money, but also a lot less than I was expecting for something like that. [1] https://www.protocol.com/internet-archive-preserving-future : "The…

I love that you have a citation for 2 * 100 million = 200 million. I've seen papers where the use of "=" sweeps a lot of non-trivial equalities under the rug, but this is the first time I've seen something go the other way this far.

I suppose the claim is rather shocking and warrants citations - $200m to host the entire internet archive forever? I don't blame you for the excessive citation.

Post reply on HN