Live data from Hacker News

Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

cloud.google.com

71–80 of 158 posts

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#71
post #7

Earlier quoted context omitted.

There's no interesting distinction between a "native" transport protocol and a transport protocol running on top of a UDP shim. The UDP header is probably necessary for ECMP.

? Maybe i'm misunderstanding what your trying to say, but there are major differences between an ethernet + IP transport and other transports like fiber channel (or even token ring, atm, etc) which has built into the lowest layers buffer crediting / flow control, retransmission, prioritization, etc. Sure you can build much of that higher in the stack but it requires everything in the network to be playing the same ga…

By transport protocol I mean layer 4. FC/TR/ATM/IB are (mostly) layer 2 protocols.

I think the idea is that Falcon assumes the underlying network is semi-crappy and works around that (e.g. Falcon assumes that packets arrive out of order then it puts them back in order).

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#72
post #55

Earlier quoted context omitted.

It's not upside down. RDMA and NVMe run on top of Falcon which runs on top of UDP.

RDMA is a low-level physical transport. You are saying that they are going to emulate RDMA on top of Falcon. Are they going to run Ethernet and then IP on top of that? The diagram is confusing since it upside down to layering direction, but the article is clear. Falcon is a hardware transport protocol, replacing Ethernet. Like Ethernet, it runs on top of physical transport like RDMA. And IP runs on top of Falcon and…

RDMA is a low-level physical transport.

It's not. For example, RDMA can run on top of TCP (iWarp). NVMe can also run on TCP. Now replace TCP with Falcon.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#73

Hmmm, so much of this looks like an attempt to solve the problems that were solved with fibre channel a couple decades back. Which I guess is standard NIH, with the advantage of not having to pay the FC consortium 95% HW margins. But still, you would think that some of those lessons could be learned before replacing it. AKA FC routes IP as one of its many protocols on top of the lower levels providing far more servic…

Maybe a better example would be Infiniband which is a simple and efficient protocol... but it's basically owned by Nvidia. For whatever reason Broadcom won't make Infiniband ASICs and Google doesn't want to be locked in to Nvidia so they have to use Ethernet.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#75
post #73

Hmmm, so much of this looks like an attempt to solve the problems that were solved with fibre channel a couple decades back. Which I guess is standard NIH, with the advantage of not having to pay the FC consortium 95% HW margins. But still, you would think that some of those lessons could be learned before replacing it. AKA FC routes IP as one of its many protocols on top of the lower levels providing far more servic…

Maybe a better example would be Infiniband which is a simple and efficient protocol... but it's basically owned by Nvidia. For whatever reason Broadcom won't make Infiniband ASICs and Google doesn't want to be locked in to Nvidia so they have to use Ethernet.

[deleted]

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#76

Earlier quoted context omitted.

Why though? You can't route MAC because... ? Because ipv4 provides a higher entropy address? Because MAC is self-assigned and reduplication would require a higher level system? or just because we just don't use MAC addresses that way? I'm certain there are reasons IP came to live alongside/on top of MAC, but saying you can't do multi-hop routing with it just isn't true. If all the technologies of the Internet were re…

0C:F9:31:D2:DB:51 AB:33:C6:C6:19:74 I used a MAC address generator to get those two, but I think two is enough to make the discussion. Current reality aside, would you be able to identify those with binary math as being on the same network device, different network devices, across the world? MAC addresses on physical NICs are provided by the manufacturer, sure you can adjust them but I think that leaves the good-fait…

So it seems like the primary use of IP, as you describe, is to define a way to narrow the search to sub address groups so as to not require enumerating every address in the scheme.

Still, there's doesn't seem to be any reason you couldn't just say "device 1 gets MAC 00:00:00:00:00:01" and "device 2 gets 00:00:00:00:00:02" and the gateway controller gets :::00 and there's a special address on :::FF that can be used to talk to everyone...

Is that it? Is that all there is to IP? A loose pattern for reducing search scope, a couple reserved addresses for special cases, and a balance between address bitsize and total number of unique addresses (without requiring additional routing complexity)?

It all seems so... simple

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#77
post #46

Earlier quoted context omitted.

But you can't route MACs... you can't associate domain names to MACs... Maybe for a LAN that's fine, but for connecting devices between broadcast domains, it just doesn't cut it.

Why though? You can't route MAC because... ? Because ipv4 provides a higher entropy address? Because MAC is self-assigned and reduplication would require a higher level system? or just because we just don't use MAC addresses that way? I'm certain there are reasons IP came to live alongside/on top of MAC, but saying you can't do multi-hop routing with it just isn't true. If all the technologies of the Internet were re…

Because aggregation, summarization and continents are a thing. Also... there are things which speak IP and don't use Ethernet for underlying communications, specifically in the network carrier and high performance optical space.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#78
post #33

Earlier quoted context omitted.

1. Your forwarding table would have to be larger because Ethernet uses exact match instead of longest prefix match. For example, you might be limited to 128K servers total while Google has millions. 2. The Ethernet header has less entropy for ECMP than a UDP/IP header. Maybe you could add entropy somewhere but ASICs may not support it. 3. You're breaking compatibility with... everything. Maybe Google could afford thi…

1. In a single data center scale computing system, though? 2. Use some bytes outside of the header? 3. I get the impression this needs hardware to really use well anyway.

2. Don’t do this. It causes reordering between flows and software doesn’t like out of order packets.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#79

Earlier quoted context omitted.

0C:F9:31:D2:DB:51 AB:33:C6:C6:19:74 I used a MAC address generator to get those two, but I think two is enough to make the discussion. Current reality aside, would you be able to identify those with binary math as being on the same network device, different network devices, across the world? MAC addresses on physical NICs are provided by the manufacturer, sure you can adjust them but I think that leaves the good-fait…

So it seems like the primary use of IP, as you describe, is to define a way to narrow the search to sub address groups so as to not require enumerating every address in the scheme. Still, there's doesn't seem to be any reason you couldn't just say "device 1 gets MAC 00:00:00:00:00:01" and "device 2 gets 00:00:00:00:00:02" and the gateway controller gets :::00 and there's a special address on :::FF that can be used to…

Then how do you send traffic to device1 on another network? You need globally unique addresses and hierarchy. Go back to the drawing board and come back when you’ve ended up inventing a worse IP protocol.

> It all seems so... simple

Because you haven’t even thought through basic use cases.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#80

Earlier quoted context omitted.

Ah yes, instead of going to google.com or 192.168.1.1 or adding a printer connected to my Wi-Fi, let me open my big yellow pages of globally unique MAC addresses…

I think op was just saying as a bus on a local network for some very specialized application.

But you can do that already. DNS, ARP, TCP/UDP, and IP are all completely optional.

Just write a program to send and receive Ethernet frames. I believe it doesn’t have to contain IP.

Post reply on HN