Live data from Hacker News

Internet Archive as a default host-of-record for startups

twitter.com

121–130 of 188 posts

Re: Internet Archive as a default host-of-record for startups

#121

Earlier quoted context omitted.

It costs the Internet Archive $2/GB to host content in perpetuity. They have a tool, Archive It, that will periodically crawl your site for archival purposes if you are not technical. For my needs, I run a report monthly for the content I’ve archived using my IA account to determine archived GBs, and then donate the amount needed to cover those costs. Consider reaching out to their patron services email address with…

> It costs the Internet Archive $2/GB to host content in perpetuity. Do you have source/more info than that? Lets say the internet archive is 100 PB [1], that's 100,000,000 GB [2], and at that rate it comes out to $200 million [3] for the whole thing forever. That's a lot of money, but also a lot less than I was expecting for something like that. [1] https://www.protocol.com/internet-archive-preserving-future : "The…

Re: [1] 45 PB = 45,000 TB

Re: Internet Archive as a default host-of-record for startups

#122

Imagine someone building this for SaaS hosting--a perma-Heroku, or something like it. That's actually a huge value-add. Suddenly tinyStartupA doesn't need to convince largeCorpB that it's going to be around for forever. The service can exist in perpetuity without the company. Complex repercussions obviously around acquisition, IP, and other business dimensions however. Maybe unworkable even. But I think there's a wor…

I wonder if its worth it for these platforms (like Heroku) to simply add a donation portal. It's not as future proof as something fully open, but it wouldn't need the design, implementation and maintenance of a brand new fully open PAAS ecosystem.

Re: Internet Archive as a default host-of-record for startups

#123
post #9

The feature I most want from the Internet Archive is the ability to donate them an old domain name and enough cash to renew it for the next hundred years such that they can keep an archived version of a site available (without breaking any incoming links) for a very long time. They would also need to be able to handle legal administration costs of things like DMCA take-down notices, but I assume they already have to…

> The feature I most want from the Internet Archive The feature I most want from IA is a streamlined system to delete content they have archived on domains that I own, including a proper privacy law compliance effort on their part. They have intentionally made it a difficult, manual process to get content removed. They operate as a de facto malicious crawler. They massively violate GDPR with how they operate and few…

https://web.archive.org/web/20200813235643/http://slawsonand...

> Article 3(2), a new feature of the GDPR, creates extraterritorial jurisdiction over companies that have nothing but an internet presence in the EU and offer goods or services to EU residents[1]. While the GDPR requires these companies[2] to follow its data processing rules, it leaves the question of enforcement unanswered. Regulations that cannot be enforced do little to protect the personal data of EU citizens.

> This article discusses how U.S. law affects the enforcement of Article 3(2). In reality, enforcing the GDPR on U.S. companies may be almost impossible. First, the U.S. prohibits enforcing of foreign-country fines. Thus, the EU enforcement power of fines for noncompliance is negligible. Second, enforcing the GDPR through the designated representative can be easily circumvented. Finally, a private lawsuit brought by in the EU may be impossible to enforce under U.S. law.

[snip]

> Currently, there is a hole in the GDPR wall that protects European Union personal data. Even with extraterritorial jurisdiction over U.S. companies with only an internet presence in the EU, the GDPR gives little in the way of tools to enforce it. Fines from supervisory authorities would be stopped by the prohibition on enforcing foreign fines. The company can evade enforcement through a representative simply by not designating one. Finally, private actions may be stalled on issues of personal jurisdiction. If a U.S. company completely disregards the GDPR while targeting customers in the EU, it can use the personal data of EU citizens without much fear of the consequences. While the extraterritorial jurisdiction created by Article 3(2) may have seemed like a good way to solve the problem of foreign companies who do not have a physical presence in the EU, it turns out to be practically useless.

Re: Internet Archive as a default host-of-record for startups

#124
Why is there only one IA?

Why is IA not globally distributed, like a CDN?

I use IA for "problem" websites, e.g., ones that rely on SNI, i.e., ones hosted at certain CDNs. I simply add add these sites to a list and the local proxy does the rest.

      http-request set-uri https://web.archive.org/web/1if_/http://%[req.hdr(host)]%[pathq] if { hdr(host) -m str -f list }
IA "hosts" an enormous number of sites without the need for SNI (plaintext hostnames sent over the wire).

EDIT: @sebow the way they (re)format the HTML is less friendly to the text-only browser I use.

Re: Internet Archive as a default host-of-record for startups

#125
I'd love it if the Wayback Machiine were less touchy/more reliable.

The failure mode I see very often is that the frontend apparently doesn't know what the backend's doing: The part which ingests URLs and tells you what URLs have been archived does not know what archives the backend has, so it will tell you a page has been archived and give you a link to the archive, but when you click the link, it tells you it does not have the page archived, oh, look, it exists online, would you like to archive it now? Archive it again, and it will tell you that you can only archive a page once every 45 minutes. If you're a weird little obsessive like myself, you go through this process a half-dozen times for one page before it acknowledges that, yes, it does have the page archived (once, mind you) and you can actually see it.

While I'm filing bitch reports...

The Wayback Machine apparently loves setting cookies. It will set cookies until it has exceeded its own ability to accept cookies, at which point it will give you a blank page and you have to look in the developer console to figure out that it sent you a "too many cookies" error in the response header. I've had to force my browsers to not accept any cookies from the Internet Archive to fix this.

Re: Internet Archive as a default host-of-record for startups

#126

Why is there only one IA? Why is IA not globally distributed, like a CDN? I use IA for "problem" websites, e.g., ones that rely on SNI, i.e., ones hosted at certain CDNs. I simply add add these sites to a list and the local proxy does the rest. http-request set-uri https://web.archive.org/web/1if_/http://%[req.hdr(host)]%[pathq] if { hdr(host) -m str -f list } IA "hosts" an enormous number of sites without the need f…

archive.today/.ph/etc is vastly superior for doing the intended purpose here: archive[given that you have the URL obviously]

IA is more of an curated internet archive + explorer(which granted is very good).

Re: Internet Archive as a default host-of-record for startups

#127

Why is there only one IA? Why is IA not globally distributed, like a CDN? I use IA for "problem" websites, e.g., ones that rely on SNI, i.e., ones hosted at certain CDNs. I simply add add these sites to a list and the local proxy does the rest. http-request set-uri https://web.archive.org/web/1if_/http://%[req.hdr(host)]%[pathq] if { hdr(host) -m str -f list } IA "hosts" an enormous number of sites without the need f…

What’s your issue with SNI/threat model? If you use a non-SNI site, anyone can tell which site you are visiting since there’s only one domain on that IP.

Re: Internet Archive as a default host-of-record for startups

#128
post #59

Earlier quoted context omitted.

I think he's referring to something like IPFS. https://en.wikipedia.org/wiki/InterPlanetary_File_System http://ipfs.io You can put the storage costs on the nodes because storage at archive.org's scale adds up, especially when it's run by volunteers.

It looks like you could use IPFS to accomplish this without using a blockchain.

Isn't IPFS pretty closely tied with FileCoin?

Re: Internet Archive as a default host-of-record for startups

#129
post #75

I don't understand why Carmack thinks blockchain should be a component of this. Anyone care to elaborate on how that would make this easier/better?

I stopped reading when blockchain was mentioned.

I sincerely hope that we’re only witnessing

“Any sufficiently long Internet discussion will propose blockchain as a solution.”

rather than

“Blockchain is eating the world.”

Re: Internet Archive as a default host-of-record for startups

#130

Why is there only one IA? Why is IA not globally distributed, like a CDN? I use IA for "problem" websites, e.g., ones that rely on SNI, i.e., ones hosted at certain CDNs. I simply add add these sites to a list and the local proxy does the rest. http-request set-uri https://web.archive.org/web/1if_/http://%[req.hdr(host)]%[pathq] if { hdr(host) -m str -f list } IA "hosts" an enormous number of sites without the need f…

What’s your issue with SNI/threat model? If you use a non-SNI site, anyone can tell which site you are visiting since there’s only one domain on that IP.

Seconding that curiosity. Pretty much every web server I've built uses SNI (at least if it's hosting sites under multiple domains), and the only "downside" of which I'm aware is the lack of IE6 support.
Post reply on HN