Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thank you for helping us increase our bandwidth

blog.archive.org

Post reply on HN
Print view View original

151–160 of 207 posts

Re: Thank you for helping us increase our bandwidth

#151

Post by bits_n_bytes »

jonah-archive wrote: post #146
bits_n_bytes wrote: post #136

Earlier quoted context omitted.

I understand money is tight for a non-profit, but why are bandwidth costs a gating factor? Both your upstreams, Cogent and Hurricane Electric, offer 100G ports at fiveish grand per month in carrier neutral DCs. Given that your budget is in the millions, an outlay of this magnitude doesn't seem wildly out of the question. If you can explain what the problems are in getting more bandwidth, I'd be more than happy to see…

Happy to chat in much greater detail over email if you like (in my profile) -- transit specifically is not generally our limiting cost (though of course in the context we operate in, every penny counts -- e.g. we run our own DCs, ambiently cooled, single-feed grid power with minimal backing, etc). We generally run very close to the edge, capacity-wise, in order to get the most out of what we have -- in this case, dem…

Email sent.

Re: Thank you for helping us increase our bandwidth

#152

Post by samoa42 »

EdJiang wrote: post #88
dmd wrote: post #71

Earlier quoted context omitted.

The IA is about 50 PB. IABAK stores 100 TB, or 0.2% of it.

You can get an 8TB HDD for $150 right now. That's 6250 drives. That's about $1MM in drives, which doesn't sound that cost-prohibitive. Obviously that's not the whole cost since you need to pay for bandwidth, replication and other infrastructure like the host node, but it sounds like something that could be even hosted by a number of volunteers on r/datahoarder or r/homelab. I also remember reading about Sia on HN, wh…

> That's about $1MM

is that a "Million Million" eg. 10^12 ?

Re: Thank you for helping us increase our bandwidth

#153

Post by jfim »

samoa42 wrote: post #152
EdJiang wrote: post #88

Earlier quoted context omitted.

You can get an 8TB HDD for $150 right now. That's 6250 drives. That's about $1MM in drives, which doesn't sound that cost-prohibitive. Obviously that's not the whole cost since you need to pay for bandwidth, replication and other infrastructure like the host node, but it sounds like something that could be even hosted by a number of volunteers on r/datahoarder or r/homelab. I also remember reading about Sia on HN, wh…

> That's about $1MM is that a "Million Million" eg. 10^12 ?

Some industries use M for thousand. For example, advertising uses CPM (cost per mille), which is the cost for 1000 clicks or views.

In this case, 6250*150 = 937500, almost a million.

Re: Thank you for helping us increase our bandwidth

#154

Post by lkramer »

Crazy idea: Why don't ICANN hand over the running of .org to archive.org? It's a bit outside their expertise, but they do have knowhow on running tech to scale, and the extra revenue could be used to fund a very good cause.

Also, I'd trust these guys not to mishandle .org

(yeah, I know it's not going to happen)

Re: Thank you for helping us increase our bandwidth

#155

Post by jonah-archive »

lkramer wrote: post #154

Crazy idea: Why don't ICANN hand over the running of .org to archive.org? It's a bit outside their expertise, but they do have knowhow on running tech to scale, and the extra revenue could be used to fund a very good cause. Also, I'd trust these guys not to mishandle .org (yeah, I know it's not going to happen)

> Also, I'd trust these guys not to mishandle .org

As much as I like the idea of running a TLD, if anyone gives me a TLD, I'm gonna put an MX record on it and be jonah@org. That's a promise.

EDIT: To be fair, I would probably be overwhelmed with the usual sense of responsibility and not do this. But, the temptation

Re: Thank you for helping us increase our bandwidth

#156

Post by Jleagle »

jonah-archive wrote: post #155
lkramer wrote: post #154

Crazy idea: Why don't ICANN hand over the running of .org to archive.org? It's a bit outside their expertise, but they do have knowhow on running tech to scale, and the extra revenue could be used to fund a very good cause. Also, I'd trust these guys not to mishandle .org (yeah, I know it's not going to happen)

> Also, I'd trust these guys not to mishandle .org As much as I like the idea of running a TLD, if anyone gives me a TLD, I'm gonna put an MX record on it and be jonah@org. That's a promise. EDIT: To be fair, I would probably be overwhelmed with the usual sense of responsibility and not do this. But, the temptation

Would be a nightmare with all the crappy email validators out there..

Re: Thank you for helping us increase our bandwidth

#157

Post by teekert »

TeMPOraL wrote: post #148
teekert wrote: post #147

Earlier quoted context omitted.

But I have bandwidth to share, not cash. I'm already at the lowest subscription level with my ISP. I understand the argument, but if there was a general technical solution this would be nice, right?

That's true. Does such solution exist, though? I had some hopes for Filecoin (i.e. a P2P system with monetary incentives attached to backing up other people's data, from one of the very few groups in the crypto space that don't look like scammers to me), but I haven't heard anything about it in a while.

I used to participate in such a service from Lacie (or was it Lacie?), I shared 1 TB, got 1 TB in return "cloud" storage (I think multiplied by the time you were online actually), which was in fact, distributed (after local encryption) over many other online PCs. I ran it on my server but one was not even obligated to keep their PC on. It worked "meh" (it was all java based with a meh UI) and they discontinued it.

Re: Thank you for helping us increase our bandwidth

#158

Post by BelleOfTheBall »

EdJiang wrote: post #88
dmd wrote: post #71

Earlier quoted context omitted.

The IA is about 50 PB. IABAK stores 100 TB, or 0.2% of it.

You can get an 8TB HDD for $150 right now. That's 6250 drives. That's about $1MM in drives, which doesn't sound that cost-prohibitive. Obviously that's not the whole cost since you need to pay for bandwidth, replication and other infrastructure like the host node, but it sounds like something that could be even hosted by a number of volunteers on r/datahoarder or r/homelab. I also remember reading about Sia on HN, wh…

Would it remain feasible as they scale up? Their content is growing faster and faster so the number of drives would have to rise every single month, probably by several dozen even.

Re: Thank you for helping us increase our bandwidth

#159

Post by lowwave »

afpx wrote: post #59

What's the easiest way to seed the most popular content ala torrent?

What if to have every item on archive.org also have a webtorrent. I know that webtorrent leaks your private IP, but if an item is requested a lot then it will at least save them bandwidth and be shared amongst people who are requesting the content.

Re: Thank you for helping us increase our bandwidth

#160

Post by retsibsi »

tommica wrote: post #122
gxon wrote: post #70

Earlier quoted context omitted.

Twitter ought to have some kind of strikethrough feature. Allow users to mark that they no longer stand behind a tweet without completely deleting it.

What would be the incentive to use that feature over just deleting the tweet?

Integrity or the appearance thereof -- showing that you're not trying to cover up your mistakes.
Post reply on HN