Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 5-Hour CDN

fly.io

Post reply on HN
Print view View original

61–70 of 90 posts

Re: The 5-Hour CDN

#61

Post by youngtaff »

mike_d wrote: post #21
youngtaff wrote: post #17

Some of the things they miss in the post are Cloudflare uses a customised version or Nginx, same with Fastly for Varnish (don't know about Netlify and ATS) Out of the box nginx doesn't support HTTP/2 prioritisation so building a CDN with nginx doesn’t mean you're going ti be delivering as good service as Cloudflare Another major challenge with CDNs is peering and private backhaul, if you're not pushing major traffic…

HTTP/2 prioritization is a lot of hype for a theoretical feature that yields little real world performance. When a client is rendering a page, it knows what it needs in what order to minimize blocking. The server doesn't.

Yes, which is why the browser send priorities with the requests but many servers ignore these and just server responses in what ever order suits them.

If a low priority response is served before a high priority one the page is likely to be slower to render etc.

Re: The 5-Hour CDN

#62

Post by wolf550e »

pm wrote: post #60
tptacek wrote: post #37

Earlier quoted context omitted.

I still wouldn't say we do any magic Elixir stuff; rather, our platform just happens to have a combination of features (particularly edge delivery for stuff like LiveView and zero-config private networking for clustering) that make Elixir apps sing. But we've got full-time people working on Elixir now, too; we'll see where that goes. We've still got Elixir limerence here. :)

Hey Thomas, weren’t you running Latacora last time I checked?

See:

https://twitter.com/tqbf/status/1232843986529849344

https://twitter.com/tqbf/status/1276214060036632576

https://twitter.com/tqbf/status/1288891834131939329

Re: The 5-Hour CDN

#63

Post by thraxil »

8note wrote: post #41
thaumasiotes wrote: post #38

Earlier quoted context omitted.

The thundering herd problem isn't really about high levels of traffic. To the extent that that's a problem, it's just an ordinary DOS. The thundering herd problem specifically refers to what happens if you coordinate things so that all your incoming requests occur simultaneously. Imagine that over the course of a week, you tell everyone who needs something from you "I'm busy right now; please come back next Tuesday a…

Hmm. I think of thundering Herd being about retries. All your failing requests batch up when your retry strategy sucks, then you end up really high traffic on every retry, and very little in between

Retries without jitter are indeed a common source of thundering herd problems. Even with exponential backoff, if all the clients are retrying simultaneously, they'll hammer your servers over and over. Adding jitter (just a random amount of extra delay that's different for every client+retry), they get staggered and the requests are spread out.

Re: The 5-Hour CDN

#64

Post by mbStavola »

Fly is great and I love reading their blog posts.

Just hoping they come back around on CockroachDB-- I feel like it's a match made in heaven for what they're providing.

Re: The 5-Hour CDN

#65

Post by jusssi »

> 3. Be like a game server: Ping a bunch of servers and use the best. Downside: gotta own the client. Upside: doesn't matter, because you don't own the client.

"If you can run code on it, you can own it". Your front page could just be a tiny loader js that fires off a fetch() for a zero byte resource to all your mirrors, and then proceeds to load the content from the first responder.

Re: The 5-Hour CDN

#66

Post by ithkuil »

thraxil wrote: post #63
8note wrote: post #41

Earlier quoted context omitted.

Hmm. I think of thundering Herd being about retries. All your failing requests batch up when your retry strategy sucks, then you end up really high traffic on every retry, and very little in between

Retries without jitter are indeed a common source of thundering herd problems. Even with exponential backoff, if all the clients are retrying simultaneously, they'll hammer your servers over and over. Adding jitter (just a random amount of extra delay that's different for every client+retry), they get staggered and the requests are spread out.

Jitter is one way to solve it. Request coalescing is another.

It depends on the request type. Is it cacheable? Do you require a per-client side effect? ...

Re: The 5-Hour CDN

#67

Post by arghwhat »

endymi0n wrote: post #56
sciurus wrote: post #48

Earlier quoted context omitted.

I'll echo what Simon said; we share some experiences here. There's a potential footgun, though, anyone getting started with this should know about- Request coalescing can be incredibly beneficial for cacheable content, but for uncacheable content you need to turn it off! Otherwise you'll cause your cache server to serialize requests to your backend for it. Let's imagine a piece of uncacheable content takes one second…

And to add the last big one from the trifecta: Know how to deal with cacheable data. Know how to deal with uncacheable data. But by all means, know how to keep them apart. Accidentally caching uncacheable data has lead so some of the most ugly and avoidable data leaks and compromises in recent times. If you go down the "route everything through a CDN route (that can be as easy as ticking a box in the Google Cloud Pla…

no-cache does not mean content must not be cached - in fact, it specifies the opposite!

no-cache means that the response may be stored in any cache, but cached content MUST be revalidated before use.

public means that the response may be cached in any cache even if the response was not normally cacheable, while private restricts this to only the user agent's cache.

no-store specifies that this response must not be stored in any cache. Note that this does not invalidate previous cached responses from being used.

max-age=0 can added to no-store to also invalidate old cached responses should one have accidentally sent a cacheable response for this resource. No other directives have any effect when using no-store.

Re: The 5-Hour CDN

#68

Post by [deleted] »

elithrar wrote: post #30
anonymoushn wrote: post #20

Earlier quoted context omitted.

Do you know if varnish's request coalescing allows it to send partial responses to every client? For example, if an origin server sends headers immediately then takes 10 minutes to send the response body at a constant rate, will every client have half of the response body after 5 minutes? Thanks!

I don’t know about Varnish, but having worked on other implementations, you would usually have a timeout on the initial lock (semaphore) to prevent a slow connection from impacting all clients. But this is much, much harder to do once you are already streaming the response - if the time to first byte (TTFB) is quick, but the connection is low-throughout, you can’t do much at this point. But nearly all modern implemen…

[deleted]

Re: The 5-Hour CDN

#69

Post by anonymoushn »

elithrar wrote: post #30
anonymoushn wrote: post #20

Earlier quoted context omitted.

Do you know if varnish's request coalescing allows it to send partial responses to every client? For example, if an origin server sends headers immediately then takes 10 minutes to send the response body at a constant rate, will every client have half of the response body after 5 minutes? Thanks!

I don’t know about Varnish, but having worked on other implementations, you would usually have a timeout on the initial lock (semaphore) to prevent a slow connection from impacting all clients. But this is much, much harder to do once you are already streaming the response - if the time to first byte (TTFB) is quick, but the connection is low-throughout, you can’t do much at this point. But nearly all modern implemen…

I think the nginx that members of the public can get from their package manager does not have this feature, and will force each client other than the first to either wait for the entire body to be downloaded or wait for a timeout and hit the origin in a non-cacheable request.

Re: The 5-Hour CDN

#70

Post by arghwhat »

ithkuil wrote: post #66
thraxil wrote: post #63

Earlier quoted context omitted.

Retries without jitter are indeed a common source of thundering herd problems. Even with exponential backoff, if all the clients are retrying simultaneously, they'll hammer your servers over and over. Adding jitter (just a random amount of extra delay that's different for every client+retry), they get staggered and the requests are spread out.

Jitter is one way to solve it. Request coalescing is another. It depends on the request type. Is it cacheable? Do you require a per-client side effect? ...

Request coalescing in a shared cache does not solve thundering herd, it just reduces propagation to backend services. Your cache is still subject to a thundering herd, and may be unable to keep up.

The only way to solve thundering herd - which is that a load of all requests arrive within a short timespan - is to distribute requests over larger timespan.

Reducing your herd size by having fewer requests does not solve thundering herd, but may make it bearable.

Post reply on HN