Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 5-Hour CDN

fly.io

Post reply on HN
Print view View original

21–30 of 90 posts

Re: The 5-Hour CDN

#21

Post by mike_d »

youngtaff wrote: post #17

Some of the things they miss in the post are Cloudflare uses a customised version or Nginx, same with Fastly for Varnish (don't know about Netlify and ATS) Out of the box nginx doesn't support HTTP/2 prioritisation so building a CDN with nginx doesn’t mean you're going ti be delivering as good service as Cloudflare Another major challenge with CDNs is peering and private backhaul, if you're not pushing major traffic…

HTTP/2 prioritization is a lot of hype for a theoretical feature that yields little real world performance. When a client is rendering a page, it knows what it needs in what order to minimize blocking. The server doesn't.

Re: The 5-Hour CDN

#23

Post by simonw »

anonymoushn wrote: post #20
simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

Do you know if varnish's request coalescing allows it to send partial responses to every client? For example, if an origin server sends headers immediately then takes 10 minutes to send the response body at a constant rate, will every client have half of the response body after 5 minutes? Thanks!

I don't know for certain, but my hunch is that it streams the output to multiple waiting clients as it receives it from the origin. Would have to do some testing to confirm that though.

Re: The 5-Hour CDN

#24

Post by mcintyre1994 »

tiffanyh wrote: post #16
cgarvis wrote: post #14

Earlier quoted context omitted.

just started to use them for an elixir/phoenix project. multi region with distributed nodes just works. feels almost magically after all the aws work I've done the past few years.

What’s magically? I was under the impression that fly.io today (though they are working on it) doesn’t do anything unique to make hosting elixir/Phoenix app easier. See this comment by the fly.io team. https://news.ycombinator.com/item?id=27704852

They're not doing anything special to make Elixir specifically better yet, but their private networking is already amazing for it - you can cluster across arbitrary regions completely trivially. It's a really good fit for Elixir clustering as-is even without anything specially built for it. I have no idea how you'd do multi-region clustering in AWS but I'm certain it'd be a lot harder.

Re: The 5-Hour CDN

#25

Post by jabo »

Love the level of detail that Fly's articles usually go into.

We have a distributed CDN-like feature in the hosted version of our open source search engine [1] - we call it our "Search Delivery Network". It works on the same principles, with the added nuance of also needing to replicate data over high-latency networks between data centers as far apart as Sao Paulo and Mumbai for eg. Brings with it another fun set of challenges to deal with! Hoping to write about it when bandwidth allows.

[1] https://cloud.typesense.org

Re: The 5-Hour CDN

#26

Post by ksec »

It is strange that you put a Time duration in front of CDN ( content delivery network ), because given all the recent incident with Fastly, Akamai and Bunny, I read it as 5 hours Centralised Downtime Network.

Re: The 5-Hour CDN

#28

Post by sleepy_keita »

simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

Back when I was just getting started, we were doing a lot of WordPress stuff. A client contacted us, "oh yeah, later today we're probably going to have 1000x the traffic because of a popular promotion". I had no idea what to do so I thought, I'll just set the varnish cache to 1 second, that way WordPress will only get a maximum of 60 requests per second. It worked pretty much flawlessly, and taught me a lot about the importance of request coalescing and how caches work.

Re: The 5-Hour CDN

#29

Post by skunkworker »

simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

I've implemented this manually in some golang web applications I've written. It really helps when you have an expensive cache-miss operation, as it can stack the specific requests so that once the original request is served, all of the stacked requests are served with the cached copy.

Re: The 5-Hour CDN

#30

Post by elithrar »

anonymoushn wrote: post #20
simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

Do you know if varnish's request coalescing allows it to send partial responses to every client? For example, if an origin server sends headers immediately then takes 10 minutes to send the response body at a constant rate, will every client have half of the response body after 5 minutes? Thanks!

I don’t know about Varnish, but having worked on other implementations, you would usually have a timeout on the initial lock (semaphore) to prevent a slow connection from impacting all clients.

But this is much, much harder to do once you are already streaming the response - if the time to first byte (TTFB) is quick, but the connection is low-throughout, you can’t do much at this point. But nearly all modern implementations stream the bytes to all clients immediately; they don’t try to fill the cache first (they do it simultaneously).

Some implementations might avoid fanning in too much - maintaining a smaller pool of connections rather than trying get to ”1”, but that’s ultimately a trade-off at each layer of the onion, as they can still add up.

(I worked at both Cloudflare and Google, and it was a common topic: request coalescing is a big deal for large customers)

Post reply on HN