Full disclosure: thehelix112's profile indicates that it belongs to "Directory of Security Development ( http://www.edgecast.com)" .
Self submission / promotion is not an issue here if it's not illegitimated seeded with votes.
How Verizon fixed a recent backbone provider issue
11–20 of 27 posts
Re: How Verizon fixed a recent backbone provider issue
#12Earlier quoted context omitted.
Self submission / promotion is not an issue here if it's not illegitimated seeded with votes.
Nonetheless, I think it would be better if the title had been worded such that this relationship was made clear.
Re: How Verizon fixed a recent backbone provider issue
#13Full disclosure: thehelix112's profile indicates that it belongs to "Directory of Security Development ( http://www.edgecast.com)" .
There are lots of built-in monitoring functions for the hardware interfaces. For example, the laser driver for the transmitter often drops in power before failure, so there is a hardware function built-in on the receiver side to report the avg laser power. If the power drops below a certain threshold, an alarm goes off on the monitoring software.
The author is pointing out an interesting failure mode not automatically caught. As a shot in the dark: In the receiver, the hardware is spec'd to receive 31 zeros in a row (iirc, which btw goes way back to early SONET specs). If the detector (eg) is degrading in a way that causes a particular string of zeros (or some other fixed pattern) to fail, you would see it show up much like the test that was run.
What's interesting is that the errors did not light up alarms on the monitoring. Perhaps crc errors have a threshold, and the bit pattern causing the failure was showing up statistically below the threshold.
Anyway, I'm sure there's some hardware failure-analysis guy out there who would love to look harder at that link.
Re: How Verizon fixed a recent backbone provider issue
#14I realize Verizon bought Edgecast last year but why is the post being promoted by Edgecast and not Verizon themselves? Edgecast doesn't own or operate a backbone, they are an edge provider so this is a completely separate concern. This is typical dirty Verizon/big telco PR in my opinion. Edgecast is a great CDN and company, I'm curious to see how this new ownership plays out.
Re: How Verizon fixed a recent backbone provider issue
#15The cynic in me wonders if this is blog post was written solely to serve as positive PR for Verizon in the face of their horrible transit/peering infrastructure management (ie. letting links run hot so they can charge transit providers more money for each increase in commit or letting peering ports run very hot because they don't want to spend the few thousand to add another 10G because they'd rather be paid for tran…
Re: How Verizon fixed a recent backbone provider issue
#16As a (smaller) CDN, we routinely solve these sorts of problems. Never occurred to be to blog about it in such manner. But then again, we suck at PR.
Re: How Verizon fixed a recent backbone provider issue
#17Re: How Verizon fixed a recent backbone provider issue
#18That's your fucking job, what do you want, a medal?
Seriously, the level of crap that verizon, Level 3 and AT&T put out is immense. I was trying to get a 100meg line in downtown redwood city. (this was q4 2013) at first I was told that the exchange was full. After that they said I could have bonded T3. After much screaming and shouting they decided that they would do me a favour and provision a fibre line.(bear in mind this was part of a large global account, with MPLS and other such niceties)
In the end it took 3 months of epic hassle (this was without way leave) just to get to the point of connection. after another 6 weeks of pointless meandering, I had fibre.
However because of the level of skill at the NOX it was another 3 fucking weeks to get it lit properly. (15% packet loss is not acceptable by the way)
The worst part of this is the cost, $4500 a month for a steaming pile of shite, backed by people to thick to open doors effectively.
In london this is how it went down: Phone up $provider, I want a 1 gig line please.
$provider: sure, that might be up to 90 days, pending legals and survey
Me: ok
$provider: (week later) survey is done, line should be lit in a month
$provider: oh and thats £1500 a month
In conclusion, just fuck right off, get off your arse and fucking do what we all pay you for, provide some fucking bandwidth.
Re: How Verizon fixed a recent backbone provider issue
#19As a (smaller) CDN, we routinely solve these sorts of problems. Never occurred to be to blog about it in such manner. But then again, we suck at PR.
Re: How Verizon fixed a recent backbone provider issue
#20As someone who lives in the UK, and purchases bandwidth all over the world I say this: That's your fucking job, what do you want, a medal? Seriously, the level of crap that verizon, Level 3 and AT&T put out is immense. I was trying to get a 100meg line in downtown redwood city. (this was q4 2013) at first I was told that the exchange was full. After that they said I could have bonded T3. After much screaming and shou…
edit: my past experience with CenturyLink was always great -- they provided us with extremely detailed documents of the provisioned line with loss, bandwidth tests, etc.