Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare Package Repository Mirrors

cloudflaremirrors.com

Post reply on HN
Print view View original

31–40 of 59 posts

Re: Cloudflare Package Repository Mirrors

#31

Post by exikyut »

overflow_error wrote: post #11

How is this supposed to work? (I use Arch BTW) supposed I'm in a country X, does Cloudflare basically caching the request to server that are in country X? But I'm already close to that server and I'm always cautious with mirrors that are in country X, often they were out of date :(

Repeatedly poke the maintainers of the mirror and the core developers alternately until everyone gets mad and does something about it. :P

Alternatively (/ realistically), perhaps CF would prefer the value-add of rerouting where there are newer packages instead of the singular approach of geo-routing. Definitely let them know.

I'm seeing multiple comments in this subthread, I'd apply the 2nd line above to everyone affected, if a bunch of people let CF know maybe they'll figure something out.

Re: Cloudflare Package Repository Mirrors

#32

Post by rwmj »

It would be really nice if web proxies supported this as a generic mechanism, which could be applied to Linux packages, ISOs, git commits, docker layers and more. I wrote up a very half-baked proposal for this kind of thing a while back: https://rwmj.wordpress.com/2013/09/09/half-baked-idea-conten...

Then you could set cloudflare as a proxy (or transparent proxy even?) and it could inject the objects without having to visit the endpoints.

Re: Cloudflare Package Repository Mirrors

#33

Post by viraptor »

rwmj wrote: post #32

It would be really nice if web proxies supported this as a generic mechanism, which could be applied to Linux packages, ISOs, git commits, docker layers and more. I wrote up a very half-baked proposal for this kind of thing a while back: https://rwmj.wordpress.com/2013/09/09/half-baked-idea-conten... Then you could set cloudflare as a proxy (or transparent proxy even?) and it could inject the objects without having t…

Ideally soon repos will be mirrored to ipfs nodes and that will automatically pull the hash-addressed content from the closest proxy, which may or may not be cloudflare. The protocol is already there. Some initial solutions are already there too: https://github.com/victorb/arch-mirror

Re: Cloudflare Package Repository Mirrors

#34

Post by p00f »

dtf wrote: post #22

I can't get it to work, packages upgrades show this error. Any ideas? error: failed retrieving file 'XXX' from cloudflaremirrors.com : HTTP server doesn't seem to support byte ranges. Cannot resume.

Clear your partial downloads using sudo pacman -Scc, another comment says that it doesn't support byte ranges so it can neither display progress, nor resume partial downloads

Re: Cloudflare Package Repository Mirrors

#35

Post by p00f »

rwha wrote: post #28
dennajort wrote: post #3

Is it me, or it downloads so fast that the progress output doesn't even show in the terminal ? On Archlinux.

I saw the same. My best initial guess is that this is somehow caused by cloudflare either setting the content-type header to text/plain or not setting the accept-ranges header. Or maybe both?

> not setting the accept-ranges header

Mostly that, because it doesn't support resuming partial downloads either (it says http server does not support byte ranges)

Re: Cloudflare Package Repository Mirrors

#36

Post by brightball »

james412 wrote: post #5

As much as I'd love to use CloudFlare's uniformly fast network for package update, I'd never use this simply because I already depend on them for too much stuff. It's not like letting them gather logs on my computers updating would be terrible somehow, but if it's so easy to avoid donating more logs and more dependence on them, why not do it for the sake of preserving diversity? Growing companies often exhibit a long…

You’re definitely right there. So much of what CF is doing feels like it’s going to end badly.

Re: Cloudflare Package Repository Mirrors

#37

Post by Ayesh »

SXX wrote: post #4

Is anyone from CloudFlare or Canonical reading? Please parter to make mirrors for Launchpad PPAs. They're so painfully slow around the globe and there is tons of open source projects that depend on PPAs...

I recently moved to Vietnam, and I sometimes get awfully slow connection speeds to the mirrors nearby my city, to a point that switching to US mirrors is faster. (ftp.[kr|sg].debian.org)

Re: Cloudflare Package Repository Mirrors

#38

Post by gravitas »

iforgotpassword wrote: post #30
gravitas wrote: post #15

Earlier quoted context omitted.

This is my issue with it as well - it's basically their geo-sniffing magic routing you towards the closest officially listed mirrors for that country (glorified proxy); it makes the mistake of presuming that all those mirrors listed in the Arch country-mirrorlist are created equal in terms of reliability and performance, but they're not (we have a whole web page just to track out of date mirrors).

I'd guess if this gains traction, the caching on Cloudflare's side will usually make up for this, and if it hits the actual mirror behind it, the load on that one should be lower than before. It reads like they will try to proxy multiple local mirrors so time will tell how well they will pick amongst them. I'm switching for now to see how it will perform over a longer period.

The difficulty is if the mirror itself is behind, Cloudflare's cache for that specific endpoint will be correct (functional, working) but out of date compared to another mirror.

Example: around the end of June this year the mirrors.kernel.org infra was failing to sync (DNS issues of some sort) the latest content, so your upgrade would appear to have no new packages because the stale content was a valid repo, when in fact there were many updates and other mirrors were in sync.

Re: Cloudflare Package Repository Mirrors

#39

Post by stingraycharles »

Ayesh wrote: post #37
SXX wrote: post #4

Is anyone from CloudFlare or Canonical reading? Please parter to make mirrors for Launchpad PPAs. They're so painfully slow around the globe and there is tons of open source projects that depend on PPAs...

I recently moved to Vietnam, and I sometimes get awfully slow connection speeds to the mirrors nearby my city, to a point that switching to US mirrors is faster. (ftp.[kr|sg].debian.org)

How is Docker hub from Vietnam nowadays? I lived in Cambodia from 2014-2016, and it was nearly impossible to pull Docker images which have large single layers.

Would be great to have a Cloudflare solution for this.

Re: Cloudflare Package Repository Mirrors

#40

Post by iforgotpassword »

gravitas wrote: post #38
iforgotpassword wrote: post #30

Earlier quoted context omitted.

I'd guess if this gains traction, the caching on Cloudflare's side will usually make up for this, and if it hits the actual mirror behind it, the load on that one should be lower than before. It reads like they will try to proxy multiple local mirrors so time will tell how well they will pick amongst them. I'm switching for now to see how it will perform over a longer period.

The difficulty is if the mirror itself is behind, Cloudflare's cache for that specific endpoint will be correct (functional, working) but out of date compared to another mirror. Example: around the end of June this year the mirrors.kernel.org infra was failing to sync (DNS issues of some sort) the latest content, so your upgrade would appear to have no new packages because the stale content was a valid repo, when in…

Right that makes sense, so Cloudflare would actually have to make the caching context-aware, not just classic HTTP proxying.
Post reply on HN