Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thank you for helping us increase our bandwidth

blog.archive.org

Post reply on HN
Print view View original

11–20 of 207 posts

Re: Thank you for helping us increase our bandwidth

#11

Post by highclass »

highclass wrote: post #10
SteveNuts wrote: post #3

I'm really surprised they don't use more CDN for this. Anyone know the reason why it isn't served by something like cloudflare?

At the bandwidth levels they are using they would need to use Cloudflare Enterprise, and in my experience that is way more expensive than other CDN providers. Also, since archive.org has so much content, the caching ratio is going to be very bad and kill CDN efficiency while still requiring lots of direct bandwidth. Cheap direct bandwidth in their case looks best(which is what they seem to be doing).

Another aspect is they are actually using bandwidth to max and have a cap on usage spending. No CDN I know of allows capping bandwidth usage.

The way they use bandwidth means the most efficent GB/dollar cost although at the cost of poor performance.

Re: Thank you for helping us increase our bandwidth

#12

Post by chx »

toomuchtodo wrote: post #8
chx wrote: post #7

I have revoked my donations until they close their pirate library. Sorry but let's call a spade a spade. They admit: > multiple readers can access a digital book simultaneously with the only caveat being it is borrowed for two weeks but let's face it, most value of a book comes from its first reading -- and what stops you from "borrowing" it again, anyways. It's been a gigantic disappointment for me to see them do th…

I’m sorry to hear this. I will make an additional donation on your behalf. I can appreciate how authors feel, but they must also realize copyright laws are asymmetrical (tilted heavily towards copyright owners) and in these times, many have no access to their local public library. They are essentially robbed of access to the book materials their tax dollars (which pays an author) have paid for during this time. It’s…

I also find it amusing that sofware developers do not recognize any more how we are on the same side in this fight. Imagine if we were still selling boxed software and they decided that in light of COVID-19 they just hand it out with a two week limit. It's an imperfect comparison but the current situation is not a reason to just discard the law. It might not be the best law, the place to fight that is in Congress and not on the back of authors most of whom are not in the best financial state to say the least.

Re: Thank you for helping us increase our bandwidth

#13

Post by bzb3 »

chx wrote: post #7

I have revoked my donations until they close their pirate library. Sorry but let's call a spade a spade. They admit: > multiple readers can access a digital book simultaneously with the only caveat being it is borrowed for two weeks but let's face it, most value of a book comes from its first reading -- and what stops you from "borrowing" it again, anyways. It's been a gigantic disappointment for me to see them do th…

Culture wants to be free.

Re: Thank you for helping us increase our bandwidth

#14

Post by toomuchtodo »

chx wrote: post #12
toomuchtodo wrote: post #8

Earlier quoted context omitted.

I’m sorry to hear this. I will make an additional donation on your behalf. I can appreciate how authors feel, but they must also realize copyright laws are asymmetrical (tilted heavily towards copyright owners) and in these times, many have no access to their local public library. They are essentially robbed of access to the book materials their tax dollars (which pays an author) have paid for during this time. It’s…

I also find it amusing that sofware developers do not recognize any more how we are on the same side in this fight. Imagine if we were still selling boxed software and they decided that in light of COVID-19 they just hand it out with a two week limit. It's an imperfect comparison but the current situation is not a reason to just discard the law. It might not be the best law, the place to fight that is in Congress and…

You assume authors are losing revenue from this effort. It is likely this revenue would never have been realized regardless of the Archive’s efforts. A piece of content copied doesn’t mean someone would’ve paid for it.

As an aside, many SaaS products have given away their product for free due to COVID and widespread forced WFH.

https://www.entrepreneur.com/article/347840

Re: Thank you for helping us increase our bandwidth

#15

Post by SteveNuts »

adtac wrote: post #5
SteveNuts wrote: post #3

I'm really surprised they don't use more CDN for this. Anyone know the reason why it isn't served by something like cloudflare?

Please don't make the whole internet basically cloudflare. they've banned my VPN endpoint (Hetzner server) and as a result a huge chunk of websites already don't work for me despite me having done nothing wrong. I've heard reports of Tor users being restricted as well.

Okay, any one of the other CDNs then. It seems like the bandwidth costs and equipment are prohibitively expensive for them. There must be a reason they don't use them which is what I want to know.

Re: Thank you for helping us increase our bandwidth

#16

Post by makomk »

jeromegv wrote: post #9

The wayback machine has been essential through COVID as many government just publish the numbers and data "of the day" and the only way to compare to the day before is to look at IA.

Governments aren't the only ones overwriting old information with new either. The BBC has developed an annoying habit of completely overwriting old articles about Covid-19 with new, semi-related ones, meaning that the only way to see what they were saying earlier in the pandemic is through sites like the Internet Archive.

There's also at least one outright correction that only seems to exist on the Internet Archive now: https://web.archive.org/web/20200428214306/https://www.bbc.c... "Correction 25 April 2020: An earlier version of this article incorrectly said that France had conducted just under 140,000 tests a day by 21 April. The figure of just under 140,000 refers instead to the number of tests it had carried out weekly."

Re: Thank you for helping us increase our bandwidth

#17

Post by est31 »

60 Gbit/s of continuous traffic is a lot. If I'm reading the graphs right, Wikipedia "only" has 13.4 Gbit/s of outbound traffic [1]. Of course still well below the single-digit Tbit/s traffic of large internet exchanges [2] [3], but still unexpectedly large.

[1]: adding up the outbound numbers for each datacenter: 1.888 + 8.003 + 810 + 1.958 + 807: https://grafana.wikimedia.org/d/000000605/datacenter-global-...

[2]: https://en.wikipedia.org/wiki/List_of_Internet_exchange_poin...

[3]: Not 100% correct as thanks to Corona, ix.br now has passed 10 Tbit/s peaks. https://ix.br/noticia/releases/ix-br-reaches-mark-of-10-tb-s...

Re: Thank you for helping us increase our bandwidth

#19

Post by pier25 »

jeromegv wrote: post #9

The wayback machine has been essential through COVID as many government just publish the numbers and data "of the day" and the only way to compare to the day before is to look at IA.

Wikipedia is a good source too, at least for the countries I've been following.

Re: Thank you for helping us increase our bandwidth

#20

Post by Someone1234 »

I donate to the Internet Archive (recurring). I just hope they stick to their core mission, and don't get sidetracked like Wikipedia has. They offer a valuable service and I believe will help historians in the future understand this point in the internet's life.

PS - Don't forget to pick an Amazon Smile charity and use the Smile.Amazon sub-domain. Donated almost $20 to the EFF that way.

Post reply on HN