Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Your Own CDN for Fun and Profit

pasztor.at

Post reply on HN
Print view View original

21–30 of 65 posts

Re: Building Your Own CDN for Fun and Profit

#21

Post by tpetry »

chousuke wrote: post #12
chatmasta wrote: post #6

Earlier quoted context omitted.

Anycast means there are multiple routes going to the same destination. You get the route that is the shortest path via BGP to the anycast IP (least number of BGP hops). Once you have an established TCP session via one route, it will remain established through that route, as long as that route is still the “shortest” between your IP and the anycast IP. The route will not “change” unless cloudflare changes their routin…

I think it should be clarified that "destination" refers to an IP address, not an individual host. My understanding is that anycasting means a single address corresponds to multiple hosts achieved by simply advertising it from several sources with BGP, and you will often still have multiple redundant routes to any of the individual hosts behind the anycast IP because most locations will have redundant internet links.…

Google Cloud Global Loadbalancer seems to do this fancy session mirroring because you only have one IP for the http load balancing. I am very often impressed by the GCP products.

Re: Building Your Own CDN for Fun and Profit

#22

Post by nathan_f77 »

That's a fun project. For production websites and blogs, I'm pretty happy with Netlify, CloudFlare, and CloudFront. But CloudFront charges $600 per month for custom SSL certificates [1], so you could save a lot of money by just spinning up ~10 servers in different AWS regions.

I noticed this line at the bottom of the page: "When it comes to picking a solution, I often choose the less traveled road". I don't agree with that at all, and it sounds a bit like NIH syndrome. It's always better to choose the most-traveled roads, especially in DevOps. If there's a problem, then you can join those communities and contribute to the projects.

[1] https://aws.amazon.com/cloudfront/custom-ssl-domains/

Re: Building Your Own CDN for Fun and Profit

#24

Post by davidu »

"Second, BGP routes are not that stable."

This has been disproved for close to ten years empirically and academically[1]. Route flaps generally result in convergence to the exact same destination if it has another path and is still online. If it's offline, then it's working as intended, and that's no different from a server being rotated via a DNS pool going down.

1: Quick search: https://www.google.com/search?q=tcp+anycast+paper&ie=utf-8&o...

Re: Building Your Own CDN for Fun and Profit

#25

Post by jgrahamc »

davidu wrote: post #24

"Second, BGP routes are not that stable." This has been disproved for close to ten years empirically and academically[1]. Route flaps generally result in convergence to the exact same destination if it has another path and is still online. If it's offline, then it's working as intended, and that's no different from a server being rotated via a DNS pool going down. 1: Quick search: https://www.google.com/search?q=tcp+…

That's not our experience either. BGP is fine.

But it is the case that transit and peering connections are not stable (in the sense of going up and down randomly or suddenly experiencing high levels of packet loss) and active monitoring is a must.

Re: Building Your Own CDN for Fun and Profit

#26

Post by koolba »

nathan_f77 wrote: post #22

That's a fun project. For production websites and blogs, I'm pretty happy with Netlify, CloudFlare, and CloudFront. But CloudFront charges $600 per month for custom SSL certificates [1], so you could save a lot of money by just spinning up ~10 servers in different AWS regions. I noticed this line at the bottom of the page: "When it comes to picking a solution, I often choose the less traveled road". I don't agree wit…

If you can get by with only SNI connections then you don’t have to pay the $600 per month. The $600 is for a dedicated IP that will serve a single certificate.

Re: Building Your Own CDN for Fun and Profit

#27

Post by janoszen »

nathan_f77 wrote: post #22

That's a fun project. For production websites and blogs, I'm pretty happy with Netlify, CloudFlare, and CloudFront. But CloudFront charges $600 per month for custom SSL certificates [1], so you could save a lot of money by just spinning up ~10 servers in different AWS regions. I noticed this line at the bottom of the page: "When it comes to picking a solution, I often choose the less traveled road". I don't agree wit…

"When it comes to picking a solution, I often choose the less traveled road"

I forgot to add that this applies only to R&D and hobby projects, for production setups I'm a bit more careful. :)

(I'm the author.)

Re: Building Your Own CDN for Fun and Profit

#28

Post by jgrahamc »

nathan_f77 wrote: post #22

That's a fun project. For production websites and blogs, I'm pretty happy with Netlify, CloudFlare, and CloudFront. But CloudFront charges $600 per month for custom SSL certificates [1], so you could save a lot of money by just spinning up ~10 servers in different AWS regions. I noticed this line at the bottom of the page: "When it comes to picking a solution, I often choose the less traveled road". I don't agree wit…

$600/month for a custom SSL certificate is totally ridiculous. Cloudflare Pro ($20/month) includes a certificate that works with non-SNI browsers.

Re: Building Your Own CDN for Fun and Profit

#29

Post by andrewstuart2 »

nathan_f77 wrote: post #22

That's a fun project. For production websites and blogs, I'm pretty happy with Netlify, CloudFlare, and CloudFront. But CloudFront charges $600 per month for custom SSL certificates [1], so you could save a lot of money by just spinning up ~10 servers in different AWS regions. I noticed this line at the bottom of the page: "When it comes to picking a solution, I often choose the less traveled road". I don't agree wit…

The $600/mo figure has to be considered against the type of workload your servers run, and how variable your traffic patterns are.

If you need to handle bursty traffic, you're likely going to get best value in shared tenancy services until you can fully utilize your servers. Otherwise, you will probably end up paying for idle infrastructure.

Re: Building Your Own CDN for Fun and Profit

#30

Post by dzolvd »

https://github.com/apache/incubator-trafficcontrol is an open source cache control layer (working with ATS) that has features for header rewrites, ssl, and custom urls (among others). It is built for video but can be used to cache any content. Probably a bit heavy for your use case infrastructure wise though.
Post reply on HN