Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

East Coast Internet Outage

news.ycombinator.com

Post reply on HN
Print view View original

231–240 of 265 posts

Re: East Coast Internet Outage

#231

Post by dan-robertson »

mike_d wrote: post #219
dboreham wrote: post #214

Earlier quoted context omitted.

I'm probably the only person reading this thread who owns a backhoe, and it's only a small one. There's really no material that will resist being dug through and is also practical to bury over miles and miles of fiber. There obviously are processes and measures in place to prevent "backhoe-fade" events, including armored cable, tracer wires and tape above the main cable but humans being humans things sometimes don't…

Backhoes also can't be blamed for every outage. Cars hitting poles that carry aerial fiber, people ripping up fiber thinking it is copper they can steal, boat anchors at landing sites, etc. Level 3 at one point stated 17% of all fiber cuts were from squirrels... https://web.archive.org/web/20151213214019/http://blog.level...

New York is particularly bad for having lots of arial fibre that gets taken down by car crashes or squirrels.

Re: East Coast Internet Outage

#232

Post by JacobHenner »

mroche wrote: post #24

https://twitter.com/VerizonSupport/status/135410988957298278... There is a fiber cut in Brooklyn. We have no ETR, as of yet. You can use the MY Fios app for updates. *EAG

As per WSJ this was not the cause.

https://www.wsj.com/articles/verizon-internet-outage-disrupt...

Re: East Coast Internet Outage

#233

Post by korethr »

Scoundreller wrote: post #192
fcsp wrote: post #180

Earlier quoted context omitted.

Would it maybe be an idea to, going forward, color the cables individually before putting them into the ground?

If they’re all in use and it’s point to point, might be easier connecting whatever to whatever. Then simultaneously swap line cards around as they get lit up or letting IP do its thing. Source: I have no idea.

Not as feasible as it sounds initially. I don't know if you've ever looked inside a cat-5/6/whatever networking cable[1], but if you do, you'll see that it is actually a bundle of 8 individual wires, in 4 twisted-together pairs, color coded. Those different wires carry different electrical signals, and if the various signals are not supposed to be on the various wires they are supposed to be on, the connection won't work.

So imagine if you cut a Cat6 cable in half, and then spliced the individual wires back together without regard to which wires you connected. Very likely, Pair 1 negative is connected Pair 3 negative, Pair 2 has its polarity reversed, etc, and so the network link never comes online.

At the individual device level, fiber cables are connected in pairs, in a crossover fashion. Device A Tx connects to Device B Rx, and vice versa. So with a random splicing, you would end up with pairs getting crossed or broken up, invalidating the connections and labeling of every patch panel downstream, making for yet more work than the initial work of splicing the cut bundle.

1. Not impugning your intelligence, knowledge, or character here, some people never have because they've never needed or wanted to, and that's totally okay.

Re: East Coast Internet Outage

#234

Post by eternalsculptor »

nine_zeros wrote: post #218
kevwil wrote: post #215

Imagine a world where all our enemies DIDN'T know how fragile our infrastructure is. False security, surely, but still would be somewhat comforting. :)

Firstly, America should strive to not have enemies. We fail at this very badly. Secondly, ZERO countries in the world have non-fragile infrastructure. America's infrastructure is better than most countries and definitely above average even among developed countries, except public transportation.

> Firstly, America should strive to not have enemies. We fail at this very badly.

This comment is not based in reality. Does Australia have many enemies? Does South Korea? Does Japan? Taiwan?

Re: East Coast Internet Outage

#235

Post by brown9-2 »

JacobHenner wrote: post #232
mroche wrote: post #24

https://twitter.com/VerizonSupport/status/135410988957298278... There is a fiber cut in Brooklyn. We have no ETR, as of yet. You can use the MY Fios app for updates. *EAG

As per WSJ this was not the cause. https://www.wsj.com/articles/verizon-internet-outage-disrupt...

Its wild that after ~9 hours, there is still not a clear cause:

> The company hadn’t pinned down the source of the failure as of Tuesday afternoon.

Re: East Coast Internet Outage

#236

Post by bombcar »

steve_adams_86 wrote: post #207
bombcar wrote: post #201

Earlier quoted context omitted.

Old Joke: How do you get rescued from a desert island if all you have is some networking gear? Simple! Bury a network cable and wait for the backhoe to show up. Backhoes are VERY strong and will easily go through gas pipe let alone anything that might protect fiber. And if the fiber is not where the plans say it is, even calling 811 won't save you.

Even worse is that (here in Canada at least) we have infrastructure surveyed, its GPS coords recorded, then manually entered into CAD (or similar) drawings using a lot of guess work. Things like metal detectors are used to more or less guess that a detected object is in fact a gas line or something, GPS location is recorded, then someone plays connect the dots in CAD. It's unreal. So many opportunities to introduce e…

We did some work on the foundation right next to the gas meter which had a pipe going down - which unexpectedly went off direct into the yard to who knows where. They hand dug until they were confident (still took out the power line to the garage. Twice).

Re: East Coast Internet Outage

#237

Post by Spooky23 »

jzer0cool wrote: post #135

When houses are created there are building codes to adhere to. For example, when there is a water pipe behind the walls a plumber (a good one) would also put a plate in front of it. In this way, once the walls are in place, drilling away into the wall, or nailing, would hit the metal plate, thus provides some safety mechanism. As another example, underground, in public areas are markings (or at least procedures) to i…

Fiber optics rely on obscurity to remain secure. Not far from my home, there’s at least 2-3 significant fiber optics buried in a trench next to the street in the outskirts of the city, probably 30-40 inches under the ground.

The dig safe people can and do mark them, but people tend to do stupid things.

Some highways have fiber rights of way too. You can see fiber conduit on bridges on the mass pike, for example.

Re: East Coast Internet Outage

#238

Post by late2part »

mike_d wrote: post #219
dboreham wrote: post #214

Earlier quoted context omitted.

I'm probably the only person reading this thread who owns a backhoe, and it's only a small one. There's really no material that will resist being dug through and is also practical to bury over miles and miles of fiber. There obviously are processes and measures in place to prevent "backhoe-fade" events, including armored cable, tracer wires and tape above the main cable but humans being humans things sometimes don't…

Backhoes also can't be blamed for every outage. Cars hitting poles that carry aerial fiber, people ripping up fiber thinking it is copper they can steal, boat anchors at landing sites, etc. Level 3 at one point stated 17% of all fiber cuts were from squirrels... https://web.archive.org/web/20151213214019/http://blog.level...

Idiots shooting rifles at aerial fiber...

Re: East Coast Internet Outage

#239

Post by late2part »

entangledqubit wrote: post #224
mike_d wrote: post #219

Earlier quoted context omitted.

Backhoes also can't be blamed for every outage. Cars hitting poles that carry aerial fiber, people ripping up fiber thinking it is copper they can steal, boat anchors at landing sites, etc. Level 3 at one point stated 17% of all fiber cuts were from squirrels... https://web.archive.org/web/20151213214019/http://blog.level...

One of my favorite network incident reports involved someone using aerial fiber for target practice...

I didn't see your post before mine. I've had circuits down TWICE due to this.

Re: East Coast Internet Outage

#240

Post by late2part »

dboreham wrote: post #227
mike_d wrote: post #219

Earlier quoted context omitted.

Backhoes also can't be blamed for every outage. Cars hitting poles that carry aerial fiber, people ripping up fiber thinking it is copper they can steal, boat anchors at landing sites, etc. Level 3 at one point stated 17% of all fiber cuts were from squirrels... https://web.archive.org/web/20151213214019/http://blog.level...

True but generally backbone circuits aren't aerial.

False. It is common for long haul fiber paths to have material portions delivered over aerial runs.
Post reply on HN