Has anyone any good resources for learning more about the "internet-level" infrastructure affected today and how global networks are connected?
+1
Many of the comments here presume knowledge about this stuff, and I can’t follow.
Level 3 Global Outage
191–200 of 393 posts
Re: Level 3 Global Outage
#192Massive reconvergence event in their network, causing edge router bgp sessions to bounce (due to cpu). Right now all their big peers are shutting down sessions with them to give level3s network the ability to reconverge. Prefixes announced to 3356 are frozen on their route reflectors and not getting withdrawn. Edit: if you are a Level3 customer shut your sessions down to them.
History doesn't repeat, but it rhymes .... There was a huge AT&T outage in 1990 that cut off most US long distance telephony (which was, at the time, mostly "everything not within the same area code"). It was a bug. It wasn't a reconvergence event, but it was a distant cousin: Something would cause a crash; exchanges would offload that something to other exchanges, causing them to crash -- but with enough time for th…
Is that related to the hacker's crackdown?
Re: Level 3 Global Outage
#193Massive reconvergence event in their network, causing edge router bgp sessions to bounce (due to cpu). Right now all their big peers are shutting down sessions with them to give level3s network the ability to reconverge. Prefixes announced to 3356 are frozen on their route reflectors and not getting withdrawn. Edit: if you are a Level3 customer shut your sessions down to them.
History doesn't repeat, but it rhymes .... There was a huge AT&T outage in 1990 that cut off most US long distance telephony (which was, at the time, mostly "everything not within the same area code"). It was a bug. It wasn't a reconvergence event, but it was a distant cousin: Something would cause a crash; exchanges would offload that something to other exchanges, causing them to crash -- but with enough time for th…
Fascinating. Thanks for sharing! :)
Re: Level 3 Global Outage
#194I guess that why HN was temporary unreachable from my home?
and why Cloudflare was having so many issues https://www.cloudflarestatus.com/
Re: Level 3 Global Outage
#195seems like the internet in 2020 has a diminished ability to route around damage
Yes and no.
Yes, because maybe so.
No, because he issue you're commenting on doesn't suggest that. It looks like the nature of this particular outage is such that a previous iteration of the Internet wouldn't have been any better equipped to solve this faster.
Re: Level 3 Global Outage
#196So, that's why HN is unreachable from Belgium at the moment (right when I was trying to figure a dns cache problem in Firefox,of course). An ssh tunnel through OVH/gravelines is working so far. edit: Proximus. edit2: also, Orange Mobile
It was also down from South Africa. It's luckily up now. Gasps for breath
Re: Level 3 Global Outage
#197India just lost to Russia in the final of the firstever online chess olympiad, probably due to connection issues of two of its players. I wonder if it's related to this incident and if the organizers are aware. Edit: the organizers are aware, and Russia and India have now been declared joint winner.
Thanks for the update.
Wasn't able to access HN from India earlier, but other cloudflare enabled services were accessible. I assume several Network Engineers were woken up from their Sunday morning sleep to fix the issue; if any of them is reading this, I appreciate your effort.
Re: Level 3 Global Outage
#198NANOG are talking about a CenturyLink outage and BGP flapping (AS 3356) as of 03:00 US/Pacific, AS209 possibly also affected.
AS3356 is Level 3, AS209 is CenturyLink.
https://mailman.nanog.org/pipermail/nanog/2020-August/209359...
Re: Level 3 Global Outage
#199The beginning of WWIII probably looks something like this.
Re: Level 3 Global Outage
#200This explains a lot. Initially thought my mobile phone Internet connectivity was flakey because I couldn't access HN here in Australia, whilst it's fine over wi-fi (wired Internet).
I was so pissed at Waze earlier for giving up on me in a critical moment. Then I found out I'm also unable to send iMessages, but I was curious, since I could browse the web just fine.
When something doesn't work I always assume it's a problem with my device/configuration/connection.
Who would have thought it's a global event such as the repeated Facebook SDK issues.