Has anyone any good resources for learning more about the "internet-level" infrastructure affected today and how global networks are connected?
https://mobile.twitter.com/Level3 (not an internet level, just a company :)
Level 3 Global Outage
201–210 of 393 posts
Re: Level 3 Global Outage
#202Earlier quoted context omitted.
That's fascinating. But I wonder, why don't they start over, or continue where they left off, once the internet is back?
> continue where they left off The games are timed and this pause gives a lot of thinking time. If they're allowed to talk with others during the pause, then also consulting time. > why don't they start over That would be unfair to the player who was ahead. That said, both players might still be fine with a clean rematch, because being the undisputed winner feels better. I wonder if they were asked (anonymously to pr…
Seems like one of those cases where solving a “little” issue would actually require rearchitecting the entire system.
Namely, in this case, it seems like the “right thing” is for games to not derive their ELO contributions from pure win/loss/draw scorings at all, but rather for games to be converted into ELO contributions by how far ahead one player was over the other at the point when both players stopped playing for whatever reason (where checkmate, forfeit, and game disruption are all valid reasons.) Perhaps with some Best-rank (https://www.evanmiller.org/how-not-to-sort-by-average-rating...) applied, so that games that go on longer are “more proof” of the competitive edge of the player that was ahead at the time.
Of course, in most central cases (of chess matches that run to checkmate or a “deep” forfeit), such a scoring method would be irrelevant, and would just reduce to the same data as win/loss/draw inputs to ELO would. So it’d be a bunch of effort only to solve these weird edge cases like “how does a half-game that neither player forfeited contribute to ELO.”
Re: Level 3 Global Outage
#203Massive reconvergence event in their network, causing edge router bgp sessions to bounce (due to cpu). Right now all their big peers are shutting down sessions with them to give level3s network the ability to reconverge. Prefixes announced to 3356 are frozen on their route reflectors and not getting withdrawn. Edit: if you are a Level3 customer shut your sessions down to them.
What is a reconvergence event? Is that what's described in your last sentence?
Re: Level 3 Global Outage
#204Re: Level 3 Global Outage
#205Massive reconvergence event in their network, causing edge router bgp sessions to bounce (due to cpu). Right now all their big peers are shutting down sessions with them to give level3s network the ability to reconverge. Prefixes announced to 3356 are frozen on their route reflectors and not getting withdrawn. Edit: if you are a Level3 customer shut your sessions down to them.
History doesn't repeat, but it rhymes .... There was a huge AT&T outage in 1990 that cut off most US long distance telephony (which was, at the time, mostly "everything not within the same area code"). It was a bug. It wasn't a reconvergence event, but it was a distant cousin: Something would cause a crash; exchanges would offload that something to other exchanges, causing them to crash -- but with enough time for th…
Indeed. In 2018 an Erlang telco software did break, bringing down the UK and Japan.
Re: Level 3 Global Outage
#206Has anyone any good resources for learning more about the "internet-level" infrastructure affected today and how global networks are connected?
This might help https://community.fs.com/blog/tcpip-vs-osi-whats-the-differe...
Re: Level 3 Global Outage
#207Has anyone any good resources for learning more about the "internet-level" infrastructure affected today and how global networks are connected?
No particular resource to recommend, though I first learned about it in a book by Radia Perlman, but BGP is a protocol you don't hear much about unless you work in networking, and is one of the key pieces in a lot of wide-scale outages. I'd start with that.
Re: Level 3 Global Outage
#208Has anyone any good resources for learning more about the "internet-level" infrastructure affected today and how global networks are connected?
Unfortunately, this infrastructure is at an uncanny intersection of technology, business and politics.
To learn the technical aspect of it, you can follow any network engineering certification materials or resources that delve into dynamic routing protocols, notably BGP. Inter-ISP networking is nothing but setting up BGP sessions and filters at the technical level. Why you set these up, and under what conditions is a whole different can of worms, though.
The business and political aspect is a bit more difficult to learn without practice, but a good simulacrum can be taking part in a project like dn42, or even just getting an ASN and some IPv6 PA space and trying to announce it somewhere. However, this is no substitute for actual experience running an ISP, negotiating percentile billing rates with salespeople, getting into IXes, answering peering requests, getting rejected from peering requests, etc. :)
Disclaimer: I helped start a non-profit ISP in part to learn about these things in practice.
Re: Level 3 Global Outage
#209Has anyone any good resources for learning more about the "internet-level" infrastructure affected today and how global networks are connected?
I don't know something holistic, but if you are the Wikipedia rampage sort of person, here is a good place to start:
Re: Level 3 Global Outage
#210This explains a lot. Initially thought my mobile phone Internet connectivity was flakey because I couldn't access HN here in Australia, whilst it's fine over wi-fi (wired Internet).
Yep, I had a similar experience. Sites that didn't work from my home connection worked fine on mobile. After rebooting and it persisted, I assumed it was just a DNS or routing issue since they were both connecting to different networks.