Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

cloud.google.com

Post reply on HN
Print view View original

151–158 of 158 posts

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#151

Post by dezgeg »

xorcist wrote: post #127
Animats wrote: post #97

Earlier quoted context omitted.

It's been done. There's XNS, and there's QNX networking over raw Ethernet. Both worked well. Look them up. You probably don't want to go that route, but it is technically possible on a LAN.

There's also Audio-over-Ethernet, which is still used for professional audio where latency must be held low. I believe there are other broadcast protocols that runs on Ethernet for the same reason. Fibre Channel-over-Ethernet used to be a thing too, but I haven't seen it for a while. Perhaps latency wasn't as much of an issue as people thought and it lost to iSCSI. I don't think it necessarily is bad idea to run prot…

eCPRI, or essentially radio signal over ethernet (ie. from base station indoors to radio module at top of the tower/roof) is another interesting use case (with even stricter latency requirements than audio).

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#152

Post by tuetuopay »

Dylan16807 wrote: post #147
tuetuopay wrote: post #140

Earlier quoted context omitted.

1. Especially in the datacenter. When you add VMs to the mix you get LOADS of devices to address. Add on top that a single device has multiple connections (management, internet, storage, etc), you'd run out of capacity almost instantly. 2. The point is to find some bytes that are constant for a given logical stream of related packets. Taking bytes outside of the header means taking bytes from the payload, that is by…

> Especially in the datacenter. Okay, but my point there was saying "Google has millions of servers" isn't relevant, we're not looking at the entire company. Even with a few addresses per VM, how many racks do you need to put into the same shared-compute mass? One data center is the upper limit, but it doesn't have to be the entire data center.

Back of the envelope math using modern hypervisors that can fit loads of vms in a single U

- let’s put 500 vms on a single one. with 128c256t CPUs it’s easy

- say you can fit 30 of those in a single rack (the common rack is 42U) due to power constraints

- and place 10 of those racks

That’s 500 x 30 x 10 = 150000 nodes to address. With 10 racks you already blow past the scaling limits of the common datacenter switch when it comes to MAC addresses. Here are the limits for Cisco’s Nexus 9000 series, a very common datacenter switch: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/ne...

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#153

Post by NavinF »

DanielHB wrote: post #137
mintplant wrote: post #17

Oh, I was hoping this would be something built more directly over Ethernet, rather than on top of UDP/IP (if I'm understanding the layer diagram correctly). I've been working with Ethernet devices a lot lately, using the network as a communication bus, essentially. I find that there's a lot of complexity that we simply don't need: ARP, DHCP, DNS... So many points of failure. We know all the devices on our LAN and the…

take a look at CBOR if you want a little more power in your payloads. It is basically binary space-efficient JSON

Did you reply to the wrong thread?

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#154

Post by fmajid »

NavinF wrote: post #113
mintplant wrote: post #17

Oh, I was hoping this would be something built more directly over Ethernet, rather than on top of UDP/IP (if I'm understanding the layer diagram correctly). I've been working with Ethernet devices a lot lately, using the network as a communication bus, essentially. I find that there's a lot of complexity that we simply don't need: ARP, DHCP, DNS... So many points of failure. We know all the devices on our LAN and the…

If you're flexible enough to forgo UDP/IP, why not use infiniband instead of ethernet? That gets rid of all the complexity you mentioned but still gives you ordered streams

Infiniband is basically a nVidia monopoly since they bought Mellanox, and the hyperscalers who already chafe at nVidia's GPU pricing power don't like it one bit, which is why they are working so hard on getting rid of it.

I don't understand enough about niche high-performance interconnects to know if CXL is a viable alternative for Infiniband where Ethernet-based solutions have too much latency.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#155

Post by namtab00 »

camtarn wrote: post #105
wayfinder wrote: post #80

Earlier quoted context omitted.

But you can do that already. DNS, ARP, TCP/UDP, and IP are all completely optional. Just write a program to send and receive Ethernet frames. I believe it doesn’t have to contain IP.

Indeed - see industrial protocols such as EtherCAT or Powerlink, which use Ethernet frames with their own non-IP protocol on top.

aren't also ATMs using a protocol of the same "species"?

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#156

Post by cereal_cable »

tuetuopay wrote: post #152
Dylan16807 wrote: post #147

Earlier quoted context omitted.

> Especially in the datacenter. Okay, but my point there was saying "Google has millions of servers" isn't relevant, we're not looking at the entire company. Even with a few addresses per VM, how many racks do you need to put into the same shared-compute mass? One data center is the upper limit, but it doesn't have to be the entire data center.

Back of the envelope math using modern hypervisors that can fit loads of vms in a single U - let’s put 500 vms on a single one. with 128c256t CPUs it’s easy - say you can fit 30 of those in a single rack (the common rack is 42U) due to power constraints - and place 10 of those racks That’s 500 x 30 x 10 = 150000 nodes to address. With 10 racks you already blow past the scaling limits of the common datacenter switch w…

Plus layer 2 switches when they don't know the destination port will flood. With so many hosts that would be absolutely horrific.

I've seen computers at moderately sized LAN parties (talking 100 nodes, far from the large or even massive events) that were literally crippled by the broadcast traffic. At some point the flooding and layer 2 discovery (ARP) would do the same as well.

Limiting the broadcast domains with layer 3 really makes the Internet possible. Sure, you can have less overhead and simply do layer 2 only, and really it is completely possible. It's just such a rare use case that it in practice isn't important enough to actually do.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#157

Post by IshKebab »

DoingIsLearning wrote: post #149
deely3 wrote: post #128

Earlier quoted context omitted.

Could you elaborate on this please?

There is a fairly amount of technical writing describing Google's use of a single repository across a lot of enginering teams. This works for them at their scale with quite a lot of internal tooling specifically to make it work. However many people defend it as _the_ solution whereas it is a way with trade offs like most solutions.

Microsoft, Facebook and Twitter all use a monorepo too. It's not just Google.

Granted, Facebook have written their own VCS and Microsoft heavily modified Git to make it usable with monorepos (but only on Windows).

Unfortunately stock Git is bad at multirepos and monorepos. When you have "hundreds of people working full time on a project" scale, stock Git doesn't have a good answer.

Re: Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem

#158

Post by rstuart4133 »

I normally like Google blog announcements, as they are usually heavy on technical details. But not this one. Quoting, the meat of it is:

> Fine-grained hardware-assisted round-trip time (RTT) measurements with flexible, per-flow hardware-enforced traffic shaping, and fast and accurate packet retransmissions, are combined with multipath-capable and PSP-encrypted Falcon connections ... flexible ordering semantics and graceful error handling ... hardware and software are co-designed to work together to help achieve the desired attributes of high message rate, low latency, and high bandwidth

So like QUIC, but designed for low latency. Maybe. There is no indication of how they achieve it if it is, nor is there a link to further details. The bulk of the article is literally name dropping. Protocol names, FAANG company names, standards organisation names. It reads like C-suite bait. "Come join us boys - all the big guys already have. So it's a sure winner."

Post reply on HN