I was today years old when I learned Hetzner has its own undersea cables.
It's not their cable.
Hetzner subsea fibre cut outage
21–30 of 94 posts
Re: Hetzner subsea fibre cut outage
#22Earlier quoted context omitted.
The term is time domain reflectometry, TDR. It used to be really expensive, and now it's filtered down into medium-cost optical adapters.
In the non-optical domain, that used to be a feature in the most expensive cable measuring gears (to the tune of €25k about 20 years ago). I had the pleasure of using one of the devices that had that when I wired my old house. Borrowed the kit from ... an institution that allowed me to get their gear for a weekend. (Yes, I knew people in there.) Having the device tell me that a given strand in a CAT5e cable was fault…
Many (low-cost) Mikrotik routers have this feature built in now: https://wiki.mikrotik.com/wiki/Manual:Interface/Ethernet#Det...
Re: Hetzner subsea fibre cut outage
#23If any part of this interests you, you must read "Mother Earth, Motherboard" by Neal Stephenson. It's epic longform from the early Wired. https://www.wired.com/1996/12/ffglass/
Then you will also like this: https://www.submarinecablemap.com
Re: Hetzner subsea fibre cut outage
#24Dumb question: I thought TCP/IP was designed to route around cut fibre cuts.
That's pretty much what happens but no amount of routing can make up for the loss of capacity when you physically take out a part of the network. You end up with less total available bandwidth and more latency.
Re: Hetzner subsea fibre cut outage
#25This isn't Hetzner's own cable, and it's not just "any" cable - it's an Alcatel-Lucent + Cinia + EU project, with 150 terabits worth of bandwidth. It's a high profile target, but it's just as likely that it's just an anchor accident.
Re: Hetzner subsea fibre cut outage
#26Dumb question: I thought TCP/IP was designed to route around cut fibre cuts.
It is, and a cut like this rarely causes a complete outage, but it depends on whether and what kind of backup connectivity your ISP has, and the connectivity of your destination. Bigger entities tend to have more backups. Smaller entities tend to have fewer backups, and often the backups are much lower bandwidth/quality, so it's typical for service to degrade.
Re: Hetzner subsea fibre cut outage
#27Can they somehow tell exactly where the cable was cut?
This video explains and show the same concept for copper cables in a very simple and hands on way: "Cheap and simple TDR using an oscilloscope and 74AC14 Schmitt Trigger Inverter" https://www.youtube.com/watch?v=9cP6w2odGUc
This very old video does it all explain better than all todays videos. The models are just so simple, no animations.
Re: Hetzner subsea fibre cut outage
#28When I lived in Cayman, the main fiber link (Maya-1[0]) was down for about a week (anchor drag, I think). Most of the island Caribbean has limited backup connectivity. The whole island only has one fiber line (around the whole western/southern Gulf of Mexico) and a crappy backup copper line to Jamaica. For a while, it was like being on a modem again. Forget about streaming anything.
This outage Hetzner is reporting isn't a big deal due to robust backups, but these things can cripple a small island outpost.
Re: Hetzner subsea fibre cut outage
#29Happens quite often to subsea cables. I was renting a number of them and every now and again you'd get a break. You can follow the NOC's update emails: Break detected Ship has been commissioned Ship is at harbor Ship has sailed out to the cut Fusing the ends Service restored
What's the typical timeline for that cycle? Hours, days, weeks? Just curious. I'm sure it costs an astronomical amount of money.
Re: Hetzner subsea fibre cut outage
#30Bitten by a shark, or snagged on a russian submarine? Whichever the case, I just ran an ansible deployment over our cluster that's spread over their Frankfurt and Helsinki datacenters, and I didn't notice any difference. I guess that's to be expected, their line between those datacenters is just an optimization, the internet is still functioning as it always is and maybe Hetzner gets a slightly larger egress bill fro…
> Bitten by a shark, or snagged on a russian submarine? In all likelihood much less interesting, probably snagged in a fishing net or hit by an anchor.
I imagined they would be hurried under the sea bed. Probably cheaper to place them on the surface?