Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 5-Hour CDN

fly.io

Post reply on HN
Print view View original

41–50 of 90 posts

Re: The 5-Hour CDN

#41

Post by 8note »

thaumasiotes wrote: post #38
cortesoft wrote: post #31

Earlier quoted context omitted.

"Thundering herd" problem is how I have always heard it called.

The thundering herd problem isn't really about high levels of traffic. To the extent that that's a problem, it's just an ordinary DOS. The thundering herd problem specifically refers to what happens if you coordinate things so that all your incoming requests occur simultaneously. Imagine that over the course of a week, you tell everyone who needs something from you "I'm busy right now; please come back next Tuesday a…

Hmm. I think of thundering Herd being about retries.

All your failing requests batch up when your retry strategy sucks, then you end up really high traffic on every retry, and very little in between

Re: The 5-Hour CDN

#42

Post by 33degrees »

intricatedetail wrote: post #34

Does Nginx still not support cache invalidation? If you setup long TTL, is there a way to remove some files from cache without nuking entire cache and restarting an instance?

It's supported, but only for NGINX Plus. You can kind of work around it by using proxy_cache_bypass though

Re: The 5-Hour CDN

#43

Post by mrkurt »

cortesoft wrote: post #33

The hard part of building a CDN is not setting up an HTTP cache, it is setting up an HTTP cache that can serve thousands of different customers.

Making a service multitenant is more complex, yes. But many companies roll their own CDNs. There are lots of good reasons to do that, and it's a problem that can be reduced to a single developer for understanding.

Re: The 5-Hour CDN

#44

Post by mrkurt »

lupire wrote: post #39
vmception wrote: post #18

>The term "CDN" ("content delivery network") conjures Google-scale companies managing huge racks of hardware, wrangling hundreds of gigabits per second. But CDNs are just web applications. That's not how we tend to think of them, but that's all they are. You can build a functional CDN on an 8-year-old laptop while you're sitting at a coffee shop. huh yeah never thought about it I blame how CDNs are advertised for the…

It's misleading. CDN software might be simple in the basic happy case, but you still need a Network of nodes to Deliver the Content.

Well it's a self serving article! It's easy to turn up a network of nodes on Fly.io. It's a little harder, but not impossible, to do the same elsewhere.

Re: The 5-Hour CDN

#45

Post by mrkurt »

tshaddox wrote: post #3
legrande wrote: post #2

I like to blog from the raw origin and not use CDNs because if a blogpost is changed I have to manually purge the CDN cache, which can happen a lot. Also CDNs have the caveat in that if they're down, it can make a page load very slow since it tries to load the asset.

If you’re okay with every request having the latency all the way to your origin, you can have the CDN revalidate its cache on every request. Your origin can just check date_updated (or similar) on the blog post to know if the cache is still valid without needing to do any work to look up and render the whole post. To further reduce load and latency to your origin, you can use stale-while-revalidate to allow the CDN t…

We've seen people use background revalidation to great effect, particularly in front of S3. You can get pretty close to one stale request per cache entry this way. And if-modified-since requests are really cheap.

Re: The 5-Hour CDN

#46

Post by mrkurt »

jabo wrote: post #25

Love the level of detail that Fly's articles usually go into. We have a distributed CDN-like feature in the hosted version of our open source search engine [1] - we call it our "Search Delivery Network". It works on the same principles, with the added nuance of also needing to replicate data over high-latency networks between data centers as far apart as Sao Paulo and Mumbai for eg. Brings with it another fun set of…

I'd love to read about it.

Re: The 5-Hour CDN

#47

Post by dilyevsky »

spondyl wrote: post #13
cj wrote: post #8

Earlier quoted context omitted.

> If you’re okay with every request having the latency all the way to your origin, you can have the CDN revalidate its cache on every request. It's also worth mentioning that even when revalidating on every request (or not caching at all), routing through a CDN can still improve overall latency because the TLS can be terminated at a local origin server, significantly shortening the TLS handshake.

Ah, the TLS shortening aspect of a CDN is something that seems obvious in hindsight but I'd never really thought about it. Thanks!

Not just tls but generally tcp will slowstart faster on lower rtt connection (and edge can keep origin connection always open so it stays “warm”)

Re: The 5-Hour CDN

#48

Post by sciurus »

simonw wrote: post #12

This article touches on "Request Coalescing" which is a super important concept - I've also seen this called "dog-pile prevention" in the past. Varnish has this built in - good to see it's easy to configure with NGINX too. One of my favourite caching proxy tricks is to run a cache with a very short timeout, but with dog-pile prevention baked in. This can be amazing for protecting against sudden unexpected traffic spi…

I'll echo what Simon said; we share some experiences here. There's a potential footgun, though, anyone getting started with this should know about-

Request coalescing can be incredibly beneficial for cacheable content, but for uncacheable content you need to turn it off! Otherwise you'll cause your cache server to serialize requests to your backend for it. Let's imagine a piece of uncacheable content takes one second for your backend to generate. What happens if your users request it at a rate of twice a second? Those requests are going to start piling up, breaking page loads for your users while your backend servers sit idle.

If you are using Varnish, the hit-for-miss concept addresses this. However, it's easy to implement wrong when you start writing your own VCL. Be sure to read https://info.varnish-software.com/blog/hit-for-miss-and-why-... and related posts. My general answer to getting your VCL correct is writing tests, but this is a tricky behavior to validate.

I'm unsure how nginx's caching handles this, which would make me nervous using the proxy_cache_lock directive for locations with a mix of cacheable and uncacheable content.

Re: The 5-Hour CDN

#49

Post by shitRETARDSsay »

babelfish wrote: post #4

fly.io has a fantastic engineering blog. Has anyone used them as a customer (enterprise or otherwise) and have any thoughts?

I run my super quantum machine learning model built on Rust on fly and it found a way to another dimension. This totally isn't an ad like the other replies. Trust me.

Re: The 5-Hour CDN

#50

Post by thaumasiotes »

8note wrote: post #41
thaumasiotes wrote: post #38

Earlier quoted context omitted.

The thundering herd problem isn't really about high levels of traffic. To the extent that that's a problem, it's just an ordinary DOS. The thundering herd problem specifically refers to what happens if you coordinate things so that all your incoming requests occur simultaneously. Imagine that over the course of a week, you tell everyone who needs something from you "I'm busy right now; please come back next Tuesday a…

Hmm. I think of thundering Herd being about retries. All your failing requests batch up when your retry strategy sucks, then you end up really high traffic on every retry, and very little in between

Where does your perspective differ from what I said above?
Post reply on HN