Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 5-Hour CDN

fly.io

Post reply on HN
Print view View original

51–60 of 90 posts

Re: The 5-Hour CDN

#51

Post by daniel_iversen »

Years ago I was involved with some high performance delivery of a bunch of newspapers, and we used Squid[1] quite well. One nice thing you could do as well (but it's probably a bit hacky and old school these days) was to "open up" only parts of the web page to be dynamic while the rest was cached (or have different cache rules for different page components)[2]. With some legacy apps (like some CMS') this can hugely improve performance while not sacrificing the dynamic and "fresh looking" parts of the website.

[1] http://www.squid-cache.org/ [2] https://en.wikipedia.org/wiki/Edge_Side_Includes

Re: The 5-Hour CDN

#52

Post by eyelidlessness »

cortesoft wrote: post #31
simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

"Thundering herd" problem is how I have always heard it called.

Thundering herd is about mitigating a problem with backpressure scenarios. If you have a backoff and a delayed queue of requests, letting them all proceed at once when the backpressure scenario resolves is likely to recreate it/create a new one. Staggering them so they proceed slightly off in time avoids that.

Re: The 5-Hour CDN

#53

Post by mnutt »

anonymoushn wrote: post #20
simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

Do you know if varnish's request coalescing allows it to send partial responses to every client? For example, if an origin server sends headers immediately then takes 10 minutes to send the response body at a constant rate, will every client have half of the response body after 5 minutes? Thanks!

Varnish has defaulted to streaming responses since varnish 4. I think it gets used for a lot of video streaming use cases.

Re: The 5-Hour CDN

#54

Post by philsnow »

simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

unrelated to CDNs but IIRC vitess did/does query coalescing too -- if it starts to serve a query for "select * from users where id = 123" and then another 20 connections all want the same query result, vitess doesn't send all 21 select queries to the backend, it sends the first one and then has all the connections wait on the backend response, then serves the same response to them all.

Re: The 5-Hour CDN

#55

Post by 3np »

As someone who’s mostly clueless about BGP but have a fair grasp of all the other layers mentioned, I’d love to see posts like this going more in depth on it for folks like myself.

Re: The 5-Hour CDN

#56

Post by endymi0n »

sciurus wrote: post #48
simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

I'll echo what Simon said; we share some experiences here. There's a potential footgun, though, anyone getting started with this should know about- Request coalescing can be incredibly beneficial for cacheable content, but for uncacheable content you need to turn it off! Otherwise you'll cause your cache server to serialize requests to your backend for it. Let's imagine a piece of uncacheable content takes one second…

And to add the last big one from the trifecta:

Know how to deal with cacheable data. Know how to deal with uncacheable data. But by all means, know how to keep them apart.

Accidentally caching uncacheable data has lead so some of the most ugly and avoidable data leaks and compromises in recent times.

If you go down the "route everything through a CDN route (that can be as easy as ticking a box in the Google Cloud Platform backend), make extra sure to flag authenticated data as cache-control: private / no-cache.

Re: The 5-Hour CDN

#57

Post by corobo »

babelfish wrote: post #4

fly.io has a fantastic engineering blog. Has anyone used them as a customer (enterprise or otherwise) and have any thoughts?

I read their blogs and I visit their site every new project I start but it just hasn't clicked with me yet.

Tinkering has been great but the addon style pricing scares the jeebs out of me (my wallet), I just assume I can't afford it for now and spin up a DO droplet. The droplet is probably more expensive for my use case but call it ADHD tax haha, at least it's capped

Re: The 5-Hour CDN

#58

Post by Akronymus »

sciurus wrote: post #48
simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

I'll echo what Simon said; we share some experiences here. There's a potential footgun, though, anyone getting started with this should know about- Request coalescing can be incredibly beneficial for cacheable content, but for uncacheable content you need to turn it off! Otherwise you'll cause your cache server to serialize requests to your backend for it. Let's imagine a piece of uncacheable content takes one second…

Speaking of non-cacheable data:

https://arstechnica.com/gaming/2015/12/valve-explains-ddos-i...

Caching is HARD.

Re: The 5-Hour CDN

#59

Post by toredash »

33degrees wrote: post #42
intricatedetail wrote: post #34

Does Nginx still not support cache invalidation? If you setup long TTL, is there a way to remove some files from cache without nuking entire cache and restarting an instance?

It's supported, but only for NGINX Plus. You can kind of work around it by using proxy_cache_bypass though

Or delete the file in question on disc, the full path is encoded, and I've come across scripts and lua modules that does it for you.

Re: The 5-Hour CDN

#60

Post by pm »

tptacek wrote: post #37
tiffanyh wrote: post #16

Earlier quoted context omitted.

What’s magically? I was under the impression that fly.io today (though they are working on it) doesn’t do anything unique to make hosting elixir/Phoenix app easier. See this comment by the fly.io team. https://news.ycombinator.com/item?id=27704852

I still wouldn't say we do any magic Elixir stuff; rather, our platform just happens to have a combination of features (particularly edge delivery for stuff like LiveView and zero-config private networking for clustering) that make Elixir apps sing. But we've got full-time people working on Elixir now, too; we'll see where that goes. We've still got Elixir limerence here. :)

Hey Thomas, weren’t you running Latacora last time I checked?
Post reply on HN