Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Surpassing 10Gb/S over Tailscale

tailscale.com

Post reply on HN
Print view View original

61–70 of 84 posts

Re: Surpassing 10Gb/S over Tailscale

#61

Post by wmf »

renewiltord wrote: post #58
ffk wrote: post #31

Earlier quoted context omitted.

Interestingly, the fastest CPU based network switches tend to do full kernel bypass. The kernel is generally slow compared to OVS and VPP, especially when they traverse over something like DPDK.

Are there consumer (<$2k) network switches that can do Wireguard in a very fast path?

By their nature as L2/L3 devices, I wouldn't expect switches to ever support Wireguard. I also haven't heard of any hardware Wireguard yet. The fastest implementation so far might be TNSR which just squeaks in under $2,000.

Re: Surpassing 10Gb/S over Tailscale

#62

Post by ffk »

arghwhat wrote: post #52
ffk wrote: post #31

Earlier quoted context omitted.

Interestingly, the fastest CPU based network switches tend to do full kernel bypass. The kernel is generally slow compared to OVS and VPP, especially when they traverse over something like DPDK.

Kernel bypass in DPDK grants the application direct access to DMA buffers so that the kernel is no longer involved. This is not because the kernel is slow, but because many small syscalls are expensive and putting your entire app in the kernel is a bad idea. There is no kernel bypass in wireguard-go, just a user-space implementation fast implementation with smart use of syscalls to minimize the overhead of being spli…

It depends on what you are trying to do though. I don’t think the kernel has an easy path to operating on a set of packet headers as a vector at this point. Not saying it can’t happen, but it’s an area where user space is already ahead.

For reference, there was a previous test that demonstrated 40gbps with ipsec between two pods on separate nodes in k8s where the encap/decap achieved 40gbps which was the line rate for the Intel NICs used.

Details were published here: https://medium.com/fd-io-vpp/getting-to-40g-encrypted-contai...

I do agree that io_uring will negate the need for DPDK for many use cases though, it will likely be a much simpler path and more secure path than DPDK.

Re: Surpassing 10Gb/S over Tailscale

#63

Post by ffk »

ilyt wrote: post #59
ffk wrote: post #31

Earlier quoted context omitted.

Interestingly, the fastest CPU based network switches tend to do full kernel bypass. The kernel is generally slow compared to OVS and VPP, especially when they traverse over something like DPDK.

It's not "kernel is slow", kernel when left to its own devices is plenty fast, the reason is that when you want to make decision about packet in userspace (vs telling kernel what to do with it via various interfaces) that kernel logic would just be overhead. It's similar for applications; if you can, say, decode whole DNS packet in one go, you don't really want kernel to spend time decoding UDP packet, then you decod…

There are some applications where the ability to vectorize the headers and operate on them with SIMD help. These types of apps tend to pin a full core to do only packet processing though. Also, syscall are expensive. A lot of work is going into making the APIs async while avoiding syscalls.

Re: Surpassing 10Gb/S over Tailscale

#65

Post by pyth0 »

ilyt wrote: post #60
api wrote: post #40

Earlier quoted context omitted.

For most compiled languages or languages with very good VMs like Java benchmarks are really testing the quality of the implementation and the depth of the implementor's understanding. I'd bet that very good Go and Rust programmers could probably converge to almost identical performance. What I wouldn't be on is that Go could equal Rust in the area of small memory footprint or on small devices.

> I'd bet that very good Go and Rust programmers could probably converge to almost identical performance. I'd imagine probably not purely because Rust uses LLVM which is VERY good at optimizing, while Go compiler is simpler and made for speed of compilation first. If Go got LLVM frontend yeah, maybe > What I wouldn't be on is that Go could equal Rust in the area of small memory footprint or on small devices. Well, Go…

> If Go got LLVM frontend yeah, maybe

While Go probably wont get an official LLVM frontend, the TinyGo project [1] is trying to bring Go to embedded systems and it does use LLVM. Unfortunately I couldn't find any use for it in a project since it lacks so many features from mainline Go. Maybe I'll check back in a few years.

[1] https://tinygo.org/

Re: Surpassing 10Gb/S over Tailscale

#66

Post by withinboredom »

2bluesc wrote: post #33
yjftsjthsd-h wrote: post #30

Earlier quoted context omitted.

If you have 10 nodes, you should already be automating with ansible/chef/puppet/whatever, at which point adding another link config is easy.

For servers sure, but things like `tailscale` exist to save every laptop and cell phone from looking like a devops project. Furthermore you could extend this argument almost every other cloud service with a primary feature of "convenience" and/or "management". Just build everything yourself.

a hub-and-spoke model comes to mind, instead of a mesh. That would probably help a lot with the management aspect, at least for consumer devices.

Re: Surpassing 10Gb/S over Tailscale

#67

Post by dtaht »

raggi wrote: post #44
red0point wrote: post #29

What‘s missing from all these figure is the resulting latency. It‘s often the case that vendors show impressive throughput numbers, but then the latency is terrible at that throughput. Do you have those numbers as well?

We do look at them to check on how we're doing, and I want to dig into this area more over time. In particular we don't do classful prioritization right now, which if you look at the typical tests for this they're often focused on multi-flow classifications. We also don't set specific congestion algorithms on our interfaces right now - availability is variable, as is the cost of them. You can see in the post here tha…

I look forward to more, at a longer RTT.

Re: Surpassing 10Gb/S over Tailscale

#69

Post by snuxoll »

wmf wrote: post #61
renewiltord wrote: post #58

Earlier quoted context omitted.

Are there consumer (<$2k) network switches that can do Wireguard in a very fast path?

By their nature as L2/L3 devices, I wouldn't expect switches to ever support Wireguard. I also haven't heard of any hardware Wireguard yet. The fastest implementation so far might be TNSR which just squeaks in under $2,000.

Really depends on what you consider a “switch” to be. Most of Mikrotik’s CRS series supports full fat RouterOS, which includes wireguard support. Though the CPU on the CRS line is much cheaper than the proper routers (CCR series), so if you’re trying to do much more than a basic firewall and NAT on a residential connection (can probably handle 1Gbps fine on most of them) performance will not be great (even my CCR2004 can only handle ~3Gbps of IPSEC traffic).

Re: Surpassing 10Gb/S over Tailscale

#70

Post by hardwaresofton »

psanford wrote: post #27
unixhero wrote: post #15

Earlier quoted context omitted.

Nebula is a Tailscale clone

Nebula predates tailscale.

link for the lazy: https://github.com/slackhq/nebula

Also with no ill intent looks like tailscale has the far more effective marketing organization :)

Post reply on HN