Isn't HN on Cloudflare? How are we reading about a CF outage on a site that runs behind CF?
The most surefire way to know if a site is behind Cloudflare (orange cloud is on) is by hitting /cdn-cgi/trace (e.g. https://news.ycombinator.com/cdn-cgi/trace ) which is the debug output from Cloudflare’s HTTP server. There’s no way to my knowledge that route can be disabled or overriden. Anyway, no, HN is not on Cloudflare, at least at the moment.
Route Leak Impacting Cloudflare
151–160 of 164 posts
Re: Route Leak Impacting Cloudflare
#152[deleted]
Re: Route Leak Impacting Cloudflare
#153Earlier quoted context omitted.
The most surefire way to know if a site is behind Cloudflare (orange cloud is on) is by hitting /cdn-cgi/trace (e.g. https://news.ycombinator.com/cdn-cgi/trace ) which is the debug output from Cloudflare’s HTTP server. There’s no way to my knowledge that route can be disabled or overriden. Anyway, no, HN is not on Cloudflare, at least at the moment.
https://www.cloudflare.com/cgi-bin/trace/ is a 404
That's because it's cdn-cgi and not cgi-bin.
Re: Route Leak Impacting Cloudflare
#154Earlier quoted context omitted.
Leaking a /4 into BGP would do basically nothing unless the originator was originally advertising a /4. IP forwarding is based on the longest-prefix match. Since allocations are sized from /8 to /24, anybody actually advertising their space would not get hijacked by a /4. The leaker would just get traffic destined toward non-advertised networks.
Then my next question is: If they didn't leak a massive range, then why was it a big problem? I assume if they leaked a bad /24 it surely wouldn't be enough to take down Cloudflare and Google for everyone... no? Did they just leak tons of bad /24s or was it something else?
the smaller the prefix I announce the more it gets spread. i.e. if I would announce the whole range via /32 it would probably go trough and all sites would be down.
BUT under normal circumstances an upstream provider would filter it since it's sloppy to not do it.
Re: Route Leak Impacting Cloudflare
#155https://news.ycombinator.com/item?id=20267790 is a more recent thread on this.
Re: Route Leak Impacting Cloudflare
#156We've written this incident up: https://blog.cloudflare.com/how-verizon-and-a-bgp-optimizer-...
Thank you!
Re: Route Leak Impacting Cloudflare
#157We've written this incident up: https://blog.cloudflare.com/how-verizon-and-a-bgp-optimizer-...
Great article! A couple missing periods at the ends of paragraphs FYI.
I'm curious why so much of this lies on Verizon's shoulders. Couldn't DQE and Allegheny have implemented the exact same best practices that Verizon should have, so it never leaked to Verizon's level? And to the extent non-Verizon subscribers were affected, couldn't their ISPs have implemented the same best practices in distrusting Verizon? Is Verizon directly responsible for routing that much of global traffic?
Re: Route Leak Impacting Cloudflare
#158Earlier quoted context omitted.
You need BGP and provider independent space for your two ISPs to both announce your space. What's the alternative approach?
Don't rely on a single IP routing through multiple ISPs, use DNS.
what? this statement makes no sense from a networking perspective.
thisissue still exists if you break up your IP space, it just makes it far harder to manage.
Re: Route Leak Impacting Cloudflare
#159We've written this incident up: https://blog.cloudflare.com/how-verizon-and-a-bgp-optimizer-...
Great article! A couple missing periods at the ends of paragraphs FYI. I'm curious why so much of this lies on Verizon's shoulders. Couldn't DQE and Allegheny have implemented the exact same best practices that Verizon should have, so it never leaked to Verizon's level? And to the extent non-Verizon subscribers were affected, couldn't their ISPs have implemented the same best practices in distrusting Verizon? Is Veri…
I'm not very knowledgeable on network routing, so be warned.
But I think at some point a network peering with verizon trusts it to route things, i.e., if I as an ISP always go through verizon to deliver traffic to cloud flare then it's out of my hands the route they take.
As for downstreams adding mitigation, ideally this would happen, but I would think you should place blame proportionally to the resources and criticality. A ten person ISP won't necessarily do everything right, and it shouldn't matter that they do, since there's a small part of the internet.
Re: Route Leak Impacting Cloudflare
#160The current it stack needs a do-over. These outages already happen on accident often because of human error. Imagine the damage a state actor could inflict by targeting these large data centers. I hope that some of the newer decentralized cloud startups like dfinity or storj takes over.
>The current it stack needs a do-over.
"The network is unreliable" is a rule of thumb that was drilled into my head in network programming class.
It always has been, it always will be. Doesn't matter if it's the internet or the link between your computer and a device sitting on your desk. And it doesn't matter what the tech is.
Making the internet more resilient only increases the severity of the failure when organizations that don't understand the risk they're taking on experience network outages.
The network is unreliable.