Live data from Hacker News

Building Your Own CDN for Fun and Profit

pasztor.at

31–40 of 65 posts

Re: Building Your Own CDN for Fun and Profit

#31
post #10

Author mentions why not use Cloudflare that CDN cache is purged often. If you want to verify if it happens for your content. You can try this tool - http://cloudperf.speedchecker.xyz/cloudflare-tester.html Side effect of this tool as you might have guessed is that using it will actually prolong the time your content stays in their cache.

So one could setup an automated crawler thatbruns frequently to keep everything in cache?

Re: Building Your Own CDN for Fun and Profit

#32

That's a fun project. For production websites and blogs, I'm pretty happy with Netlify, CloudFlare, and CloudFront. But CloudFront charges $600 per month for custom SSL certificates [1], so you could save a lot of money by just spinning up ~10 servers in different AWS regions. I noticed this line at the bottom of the page: "When it comes to picking a solution, I often choose the less traveled road". I don't agree wit…

"When it comes to picking a solution, I often choose the less traveled road" I forgot to add that this applies only to R&D and hobby projects, for production setups I'm a bit more careful. :) (I'm the author.)

Ah, that makes sense!

Re: Building Your Own CDN for Fun and Profit

#33
post #30

https://github.com/apache/incubator-trafficcontrol is an open source cache control layer (working with ATS) that has features for header rewrites, ssl, and custom urls (among others). It is built for video but can be used to cache any content. Probably a bit heavy for your use case infrastructure wise though.

Interesting, although I specifically wanted to build a push CDN (where I can push the content) rather than a pull CDN (that works with an origin) to avoid the added latency with cache misses.

Re: Building Your Own CDN for Fun and Profit

#34
post #30

https://github.com/apache/incubator-trafficcontrol is an open source cache control layer (working with ATS) that has features for header rewrites, ssl, and custom urls (among others). It is built for video but can be used to cache any content. Probably a bit heavy for your use case infrastructure wise though.

Interesting, although I specifically wanted to build a push CDN (where I can push the content) rather than a pull CDN (that works with an origin) to avoid the added latency with cache misses.

Makes sense, I am enjoying looking through the source as we are moving to an ansible and hopefully dockerized deployment model.

Re: Building Your Own CDN for Fun and Profit

#35
post #24

"Second, BGP routes are not that stable." This has been disproved for close to ten years empirically and academically[1]. Route flaps generally result in convergence to the exact same destination if it has another path and is still online. If it's offline, then it's working as intended, and that's no different from a server being rotated via a DNS pool going down. 1: Quick search: https://www.google.com/search?q=tcp+…

That's not our experience either. BGP is fine. But it is the case that transit and peering connections are not stable (in the sense of going up and down randomly or suddenly experiencing high levels of packet loss) and active monitoring is a must.

Thank you to both of you, I've edited the article to clarify that point.

Re: Building Your Own CDN for Fun and Profit

#36

Earlier quoted context omitted.

"When it comes to picking a solution, I often choose the less traveled road" I forgot to add that this applies only to R&D and hobby projects, for production setups I'm a bit more careful. :) (I'm the author.)

Ah, that makes sense!

Thank you for pointing that out, I've updated my bio to reflect that. Hopefully this way it's a little less ambiguous. :)

Re: Building Your Own CDN for Fun and Profit

#37
post #31
post #10

Author mentions why not use Cloudflare that CDN cache is purged often. If you want to verify if it happens for your content. You can try this tool - http://cloudperf.speedchecker.xyz/cloudflare-tester.html Side effect of this tool as you might have guessed is that using it will actually prolong the time your content stays in their cache.

So one could setup an automated crawler thatbruns frequently to keep everything in cache?

Yes, but you would need a crawler that does so in every region, or at least know the IPs of the edge nodes on that CDN. You would probably also hit some rate limit / DDoS protection with the CDN itself.

Re: Building Your Own CDN for Fun and Profit

#38

That's a fun project. For production websites and blogs, I'm pretty happy with Netlify, CloudFlare, and CloudFront. But CloudFront charges $600 per month for custom SSL certificates [1], so you could save a lot of money by just spinning up ~10 servers in different AWS regions. I noticed this line at the bottom of the page: "When it comes to picking a solution, I often choose the less traveled road". I don't agree wit…

I think SNI is fine, all modern browsers seem to support it: https://caniuse.com/#search=sni

Re: Building Your Own CDN for Fun and Profit

#39
Fast nameservers are not as important as author suggests. But either way one extra indirection for nameservers would allow you to choose nameserver records dynamically too. And with large enough TTLs and some traffic, clients won't have to go all the way to find out the closest nameserver, essentially providing clients with the fastest one from the cache. Since redundancy is built in into DNS any nameserver with large TTL going down won't be a problem. And unlike with anycast this is much more reliable and much cheaper, since you don't have to rely on one AS and network infrastructure as a single point of failure and you don't have to even build one either. You can use as many different hosting providers as needed.
Post reply on HN