Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

East Coast Internet Outage

news.ycombinator.com

Post reply on HN
Print view View original

221–230 of 265 posts

Re: East Coast Internet Outage

#221

Post by chokeartist »

betterunix2 wrote: post #187
t0mbstone wrote: post #152

Earlier quoted context omitted.

I thought the whole point of the internet and protocols like TCP/IP is that it's supposed to be fault tolerant and traffic is supposed to automatically re-route. Seems kind of weird how a single cable being cut can cause half the country to lose internet...

The Internet is not as redundant as you might think, and in fact there is less spare capacity to go around as a result of the pandemic (a lot more time spent streaming and on video chats by a larger number of people). Internet routing is also non-trivial (convergence has not been reached in years), and when a major link goes down it can take time for routes to stabilize. Also keep in mind that the Internet itself was…

> Internet routing is also non-trivial (convergence has not been reached in years), and when a major link goes down it can take time for routes to stabilize.

You're talking out of your ass.

Source: CCIE and BGP ninja for 20 years.

Re: East Coast Internet Outage

#222

Post by bobthepanda »

jzwinck wrote: post #186
fcsp wrote: post #180

Earlier quoted context omitted.

Would it maybe be an idea to, going forward, color the cables individually before putting them into the ground?

Of course they are colored. Like this: http://www.evertopcomms.com/144-fiber-count-gyts-fiber-optic... It is still a lot of work, and so far not automated.

It is also, as a matter of practicality, going to be very difficult to find 144 colors that can‘t be mistaken for one another, particularly in conditions that i would imagine wouldn‘t have great lighting.

Re: East Coast Internet Outage

#223

Post by tombert »

mroche wrote: post #24

https://twitter.com/VerizonSupport/status/135410988957298278... There is a fiber cut in Brooklyn. We have no ETR, as of yet. You can use the MY Fios app for updates. *EAG

Weird, I live in Brooklyn, I had no issues all day.

Re: East Coast Internet Outage

#224

Post by entangledqubit »

mike_d wrote: post #219
dboreham wrote: post #214

Earlier quoted context omitted.

I'm probably the only person reading this thread who owns a backhoe, and it's only a small one. There's really no material that will resist being dug through and is also practical to bury over miles and miles of fiber. There obviously are processes and measures in place to prevent "backhoe-fade" events, including armored cable, tracer wires and tape above the main cable but humans being humans things sometimes don't…

Backhoes also can't be blamed for every outage. Cars hitting poles that carry aerial fiber, people ripping up fiber thinking it is copper they can steal, boat anchors at landing sites, etc. Level 3 at one point stated 17% of all fiber cuts were from squirrels... https://web.archive.org/web/20151213214019/http://blog.level...

One of my favorite network incident reports involved someone using aerial fiber for target practice...

Re: East Coast Internet Outage

#225

Post by hellotomyrars »

jzer0cool wrote: post #135

When houses are created there are building codes to adhere to. For example, when there is a water pipe behind the walls a plumber (a good one) would also put a plate in front of it. In this way, once the walls are in place, drilling away into the wall, or nailing, would hit the metal plate, thus provides some safety mechanism. As another example, underground, in public areas are markings (or at least procedures) to i…

Used to work for a few ISPs. 90% of the fiber cuts that were experience while I worked there were from municipal construction crews who either weren't fully paying attention when working around fiber lines, or didn't know they were there in the first place.

The lines are supposed to be known/marked to some extent but you can't avoid the human factor.

In our worst case the same city cut the fiber line in multiple places over the course of a few weeks. Our crews would finish patching one fiber break only for the same (city) crew to cut it again a few hours later.

I wasn't on the ground so I don't know exactly how they managed it, but it was a real headache for us to manage on the customer end.

Re: East Coast Internet Outage

#226

Post by eumoria »

tombert wrote: post #223
mroche wrote: post #24

https://twitter.com/VerizonSupport/status/135410988957298278... There is a fiber cut in Brooklyn. We have no ETR, as of yet. You can use the MY Fios app for updates. *EAG

Weird, I live in Brooklyn, I had no issues all day.

in northern Brooklyn with FiOS and I had issues ~11am-12pm

Re: East Coast Internet Outage

#227

Post by dboreham »

mike_d wrote: post #219
dboreham wrote: post #214

Earlier quoted context omitted.

I'm probably the only person reading this thread who owns a backhoe, and it's only a small one. There's really no material that will resist being dug through and is also practical to bury over miles and miles of fiber. There obviously are processes and measures in place to prevent "backhoe-fade" events, including armored cable, tracer wires and tape above the main cable but humans being humans things sometimes don't…

Backhoes also can't be blamed for every outage. Cars hitting poles that carry aerial fiber, people ripping up fiber thinking it is copper they can steal, boat anchors at landing sites, etc. Level 3 at one point stated 17% of all fiber cuts were from squirrels... https://web.archive.org/web/20151213214019/http://blog.level...

True but generally backbone circuits aren't aerial.

Re: East Coast Internet Outage

#228

Post by dboreham »

stickfigure wrote: post #216
dboreham wrote: post #214

Earlier quoted context omitted.

I'm probably the only person reading this thread who owns a backhoe, and it's only a small one. There's really no material that will resist being dug through and is also practical to bury over miles and miles of fiber. There obviously are processes and measures in place to prevent "backhoe-fade" events, including armored cable, tracer wires and tape above the main cable but humans being humans things sometimes don't…

You are not the only person reading this thread who owns a backhoe. Everything else is 100% correct!

Wondering what the ad targeting AI is making of this thread..

Re: East Coast Internet Outage

#229

Post by throw0101a »

nine_zeros wrote: post #218
kevwil wrote: post #215

Imagine a world where all our enemies DIDN'T know how fragile our infrastructure is. False security, surely, but still would be somewhat comforting. :)

Firstly, America should strive to not have enemies. We fail at this very badly. Secondly, ZERO countries in the world have non-fragile infrastructure. America's infrastructure is better than most countries and definitely above average even among developed countries, except public transportation.

> Firstly, America should strive to not have enemies. We fail at this very badly.

This will never be possible, even if America does nothing 'wrong' against anyone else. As long as America (flaws and all) is a better place to live than other countries, and can be used as an example of how things can be better (though not perfect), and so will be a target of powers that don't want their own people thinking about improving things.

* https://en.wikipedia.org/wiki/The_Road_to_Unfreedom

* https://www.theguardian.com/books/2018/apr/15/the-road-to-un...

There are also plenty of bullies out there, who are only checked by American power, and wouldn't mind the US being more isolationist.

There are all sorts of problems of how America interacts with the world, but there are many other countries who would do worse things IMHO. America simply existing generates enemies.

Re: East Coast Internet Outage

#230

Post by dan-robertson »

valbaca wrote: post #142
jzer0cool wrote: post #135

When houses are created there are building codes to adhere to. For example, when there is a water pipe behind the walls a plumber (a good one) would also put a plate in front of it. In this way, once the walls are in place, drilling away into the wall, or nailing, would hit the metal plate, thus provides some safety mechanism. As another example, underground, in public areas are markings (or at least procedures) to i…

Scale: wrapping all fiber optic cables in a protective metal sheath would be cost prohibitive. Whereas adding a metal sheet to prevent a house from flooding or someone from getting electrocuted makes sense. One idea, borrowed from some chainsaw safety pants, would be to wrap the cable in a fiber web sheath that tangles in whatever cutting equipment https://www.youtube.com/watch?v=r5VSivQe760

Fibre isn’t usually cut by something like a chainsaw but rather by something like a backhoe, rodent’s teeth (squirrels on overhead lines and rats if they run through utility tunnels), or a gunshot. I doubt these super-long chainsaw chaps would even stand up to a determined rodent.
Post reply on HN