Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"BGP at home": getting a DIA circuit installed at home

aaka.sh

Post reply on HN
Print view View original

91–100 of 102 posts

Re: "BGP at home": getting a DIA circuit installed at home

#91

Post by tssva »

tgsovlerkhgsel wrote: post #69

The main issue he wanted to avoid (trees falling on lines) still seems to be there. Looking at past outages, I feel like given a certain budget, you would get much better uptime with two or more diverse (i.e. not "two fibers going exactly the same path") residential providers than with one commercial one, and for anything but the most critical projects, that's the way to go IMO. Most importantly though, anything real…

They state at the beginning of the article that the main service location is a data center in NYC. This is a redundant service location. An outage would require failure of connectivity to his home and to the NYC data center.

Re: "BGP at home": getting a DIA circuit installed at home

#92

Post by candiodari »

bogomipz wrote: post #87
linsomniac wrote: post #77

Earlier quoted context omitted.

BGP is what "the Internet" uses to say what IP addresses can be reached over the various network links that connect different providers. If you had links to two different providers, you would talk BGP to them and say "You can reach this set of IP addresses via me." They then propagate that announcement on to other providers that they connect to. This is done via ASNs (Autonomous Systems Numbers), like 32150 (my old A…

>"If you had links to two different providers, you would talk BGP to them and say "You can reach this set of IP addresses via me." There is really need to do this is many cases though. If all you are looking for is connectivity redundancy you can just take the default routes from two ISPs and configure floating static routes on your router and not worry about BGP at all. >"This is done via ASNs (Autonomous Systems Nu…

> There is really need to do this is many cases though. If all you are looking for is connectivity redundancy you can just take the default routes from two ISPs and configure floating static routes on your router and not worry about BGP at all.

That only gives you outgoing redundancy. You can talk to the world with one of the links down, but the world can't necessarily talk to your IPs with the link down.

BGP lets you advertise IPs to the world and what links they're reachable on. You can announce the same IPs on multiple links to multiple ISPs. This is what makes the internet "route around damage".

Re: "BGP at home": getting a DIA circuit installed at home

#93

Post by linsomniac »

bogomipz wrote: post #87
linsomniac wrote: post #77

Earlier quoted context omitted.

BGP is what "the Internet" uses to say what IP addresses can be reached over the various network links that connect different providers. If you had links to two different providers, you would talk BGP to them and say "You can reach this set of IP addresses via me." They then propagate that announcement on to other providers that they connect to. This is done via ASNs (Autonomous Systems Numbers), like 32150 (my old A…

>"If you had links to two different providers, you would talk BGP to them and say "You can reach this set of IP addresses via me." There is really need to do this is many cases though. If all you are looking for is connectivity redundancy you can just take the default routes from two ISPs and configure floating static routes on your router and not worry about BGP at all. >"This is done via ASNs (Autonomous Systems Nu…

>Each hop does not get its own ASN tacked on

Correct if we are talking about IP hops (router 1 to router 2 to router 3), the AS does get tacked on (possibly multiple times) for each AS hop (Level-3 to Cogent to Zayo). Thanks for clarifying that.

Re: "BGP at home": getting a DIA circuit installed at home

#94

Post by bogomipz »

candiodari wrote: post #92
bogomipz wrote: post #87

Earlier quoted context omitted.

>"If you had links to two different providers, you would talk BGP to them and say "You can reach this set of IP addresses via me." There is really need to do this is many cases though. If all you are looking for is connectivity redundancy you can just take the default routes from two ISPs and configure floating static routes on your router and not worry about BGP at all. >"This is done via ASNs (Autonomous Systems Nu…

> There is really need to do this is many cases though. If all you are looking for is connectivity redundancy you can just take the default routes from two ISPs and configure floating static routes on your router and not worry about BGP at all. That only gives you outgoing redundancy. You can talk to the world with one of the links down, but the world can't necessarily talk to your IPs with the link down. BGP lets yo…

Generally ISPs will only let you announce the same prefixes to multiple upstream ISP if you have PI address space. If you have PA address space you are probably out of luck. An ISP won't accept your prefixes if they are PA space from another ISP. At any rate you can can still use floating static routes and use a GSLB provider like Dyn to get your incoming redundancy and still not need to run BGP. In fact if you have PA address space this is your only option.

Re: "BGP at home": getting a DIA circuit installed at home

#95

Post by bcrl »

traceroute66 wrote: post #59

> While the SLA says 100%, don't expect perfection I've always considered Verizon's (and prior to that, WorldCom and/or MCI) 100% SLA to be pure marketing BS. Anyone who's ever worked with telecoms knows it's simply unrealistic. Fibres will get cut, carrier's routers will need software updates etc. etc. etc. I reckon the reason you pay a Verizon-tax is so they have spare cash floating around to pay you the inevitable…

The only thing SLAs do is tell you how much you will be compensated for downtime. If a fibre cut happens, you can end up with a 16 hour outage regardless of what the SLA guarantees. It all depends on the nature, location and extent of the damage. A couple of summers ago one of my wavelengths was down for more than 16 hours. 3 x 432 count cables had to be replaced through 3 manholes because a fibre seeking backhoe ignored the locates. Splicing high count cables takes a lot of time (even with ribbon fibre).

Re: "BGP at home": getting a DIA circuit installed at home

#96

Post by skissane »

HyperSane wrote: post #75
skissane wrote: post #10

Earlier quoted context omitted.

If I had more money than sense, I’d do the same thing. If I was filthy rich (which I’m not and probably never will be), I’d build a data centre in my mansion and buy a brand new IBM mainframe and stick it in there. Because I can. For the same reason other rich people build some huge garage with dozens of cars in it.

I would buy one of every high end enterprise storage array to play with. My former life as a VMWare admin made me appreciate storage.

Which means you'd need a IBM mainframe, since IBM mainframes require enterprise storage arrays which come with special software for emulating the old mainframe hard disks (ECKD), and if you are going to buy one of those, you'd need a mainframe to drive it.

Re: "BGP at home": getting a DIA circuit installed at home

#97

Post by HyperSane »

skissane wrote: post #96
HyperSane wrote: post #75

Earlier quoted context omitted.

I would buy one of every high end enterprise storage array to play with. My former life as a VMWare admin made me appreciate storage.

Which means you'd need a IBM mainframe, since IBM mainframes require enterprise storage arrays which come with special software for emulating the old mainframe hard disks (ECKD), and if you are going to buy one of those, you'd need a mainframe to drive it.

No, most of them use fiber channel.

Re: "BGP at home": getting a DIA circuit installed at home

#98

Post by skissane »

HyperSane wrote: post #97
skissane wrote: post #96

Earlier quoted context omitted.

Which means you'd need a IBM mainframe, since IBM mainframes require enterprise storage arrays which come with special software for emulating the old mainframe hard disks (ECKD), and if you are going to buy one of those, you'd need a mainframe to drive it.

No, most of them use fiber channel.

Yes, they do use Fibre Channel (FC), but the protocol they run on top of FC is very different from the usual FC protocols. FC has four layers, from FC-0 (the physical layer) through to FC-4 (the application layer). At level FC-4, Linux/Unix/Windows/etc servers mostly speak "Fibre Channel Protocol for SCSI" aka SCSI-FCP, which transmits SCSI commands over FC. However, while IBM mainframes do use SCSI-over-FC, they also use something else called "FICON", which instead of transmitting SCSI commands, uses the same command set as IBM's legacy non-SCSI mainframe hard drives (such as the IBM 3390, which was IBM's last non-SAN mainframe hard disk line from the late 80s / early 90s).

Mainstream platforms have what IBM calls FBA (Fixed Block Array) disks, in which the disk is an array of sectors which all have the same size. While the IBM mainframe hardware supports that, most IBM mainframe operating systems don't support those, however; Linux and VSEn are the exceptions, and z/VM is a partial exception.

As well as FBA, IBM has ECKD (Extended Count Key Data) disks, like the IBM 3390 was. (ECKD was preceded by CKD, which is conceptually the same – the same sector format, etc – but had a less efficient command set.) With ECKD, sectors can have variable sizes – each disk track can have different sized sectors, you can even mix different sector sizes within the same disk track. Also, sectors optionally have keys, and the disk has commands to search for sectors based on their keys. Originally, these variable sized sectors actually physically existed on disk; nowadays, the SAN only has standard hard disks (or SSDs), but ECKD disks are emulated in the SAN software on top of them. z/OS, the most popular IBM mainframe OS, only supports ECKD disks, not FBA ones. Linux and VSEn support both ECKD and FBA. For ECKD disks, you can't use the standard SCSI command set, you have to use IBM's DASD command set.

Many high-end enterprise arrays do support ECKD and FICON, but you usually have to pay significant extra license fees to enable the software that supports that.

Re: "BGP at home": getting a DIA circuit installed at home

#99

Post by rektide »

dboreham wrote: post #15
rektide wrote: post #8

Earlier quoted context omitted.

This is one of the most literal Conway's Law instances I've seen... > organizations design systems that mirror their own communication structure. An org for the physical layer has a box, then the org offering the connectivity layer plugs their box into the first box. Like OSI layers stacking up. https://en.m.wikipedia.org/wiki/Conway%27s_law

It's more likely to reflect a hierarchy in the network (e.g. monomode span dropping down to multimode for the demarc). The telco just colocated their box on the subscriber's premises.

I don't buy it. You seem to be contending that the second NID is just a fancy transciever between fibers. But both boxes almost certainly have QSFP or some such, could input or output whatever fiber the job calls for. Quibbling about fiber types as an excuse for having two boxes seems absurd.

Your assertion also seems contested by the words in this post:

> Dispatch 5: NID #2 install. This is where I began to see the distinction between Verizon Telecom (VzT) and Verizon Business (VzB). While the layer 1 infrastructure had been installed and tested, there still remained the IP layer, which required another dispatch for another technician to install a dedicated NID for Verizon Business -- a Ciena 3903, with 1310nm single mode optics.

It does mention a different fiber connection, but to me, this is describing each layer of the stack having both it's own organizational unit & it's own physical box that it manages.

Re: "BGP at home": getting a DIA circuit installed at home

#100

Post by HyperSane »

skissane wrote: post #98
HyperSane wrote: post #97

Earlier quoted context omitted.

No, most of them use fiber channel.

Yes, they do use Fibre Channel (FC), but the protocol they run on top of FC is very different from the usual FC protocols. FC has four layers, from FC-0 (the physical layer) through to FC-4 (the application layer). At level FC-4, Linux/Unix/Windows/etc servers mostly speak "Fibre Channel Protocol for SCSI" aka SCSI-FCP, which transmits SCSI commands over FC. However, while IBM mainframes do use SCSI-over-FC, they als…

Hearing people talk about IBM mainframes makes me realize what I sound like talking about my job to my mom. IBM could have 90% of the home computer and server market if they didn't charge so much for their stuff. They always had the best technology.
Post reply on HN