How four packets broke CenturyLink's network
theregister.co.uk
How four packets broke CenturyLink's network
1–10 of 28 posts
Re: How four packets broke CenturyLink's network
#2Correct title is: how misconfigured century link network broke when rotten packet arrived.
This title sounds like it was packet failure, while it is not, it was a matter of time until this problem occurs, hardware must be resilient to malformed input.
Re: How four packets broke CenturyLink's network
#3What protocol is that? Optional TTL sounds like the really fatal part.
Re: How four packets broke CenturyLink's network
#4Correct title is: how misconfigured century link network broke when rotten packet arrived. This title sounds like it was packet failure, while it is not, it was a matter of time until this problem occurs, hardware must be resilient to malformed input.
We can remove that ambiguity by de-baiting the title and taking out "rotten".
Re: How four packets broke CenturyLink's network
#5What protocol is that? Optional TTL sounds like the really fatal part.
Assuming that by
> 3. no expiration time, meaning that the packet would not be dropped for being created too long ago; and
they mean the TTL was set to zero.
From RFC 1812:
> A router MUST NOT originate or forward a datagram with a Time-to-Live (TTL) value of zero.
So a packet with a TTL=0 should never be on the wire (Example a router receives a packet with TTL=1, if it's not destined for that specific router, then it gets discarded). My guess is the switching vendor had bad code that didn't handle TTL=0.
Re: How four packets broke CenturyLink's network
#6From what I’ve read a lot of the reporting on this seems to use frame and packet interchangeably.
Re: How four packets broke CenturyLink's network
#7Page 7 -
>>CenturyLink and Infinera state that, despite an internal investigation, they do not know how or why the malformed packets were generated.
So we still don’t know why the rotten packets were created in the first place?
Re: How four packets broke CenturyLink's network
#8> As to what can be done to prevent similar failures, the FCC is recommending CenturyLink and other backbone providers take some basic steps, such as disabling unused features on network equipment, installing and maintaining alarms that warn admins when memory or processor use is reaching its peak, and having backup procedures in the event networking gear becomes unreachable.
Disabling unused services? Alarms when nearing resource limits? Contingency plans? How is this the first time this has come up?! These are like security & devops 101.
Re: How four packets broke CenturyLink's network
#9> As to what can be done to prevent similar failures, the FCC is recommending CenturyLink and other backbone providers take some basic steps, such as disabling unused features on network equipment, installing and maintaining alarms that warn admins when memory or processor use is reaching its peak, and having backup procedures in the event networking gear becomes unreachable. Disabling unused services? Alarms when ne…
It's kind of funny. These are best practices for running basic run of the mill web services, even something like a forum or personal homepage. Admittedly there's an obvious, massive difference in complexity, but you would expect the gold standard best practices to come from something mission critical like core Internet services and flow down to less critical services, not the other way around.
Re: How four packets broke CenturyLink's network
#10From what I’ve read a lot of the reporting on this seems to use frame and packet interchangeably.
There was a footnote in the report about that:
> In the Bureau’s discussions with Infinera, Infinera used the term “packet” to describe what some experts refer to as Ethernet frames that are sent between nodes. For the sake of simplicity, this report uses the term “packet.”