Live data from Hacker News

Surpassing 10Gb/S over Tailscale

tailscale.com

51–60 of 84 posts

Re: Surpassing 10Gb/S over Tailscale

#51
post #34

Half-way through the article it just says UDP receive coalescing, once, and never mentions it again. Do they mean interrupt mitigation? If so, using what parameters?

Author here. There was no interrupt tuning performed on the devices under test. UDP receive coalescing was enabled via the UDP_GRO sockopt.

Re: Surpassing 10Gb/S over Tailscale

#52
post #31

Pretty amazing that you can achieve such a throughput in a Golang userspace program. I wonder if other UDP based protocols like QUIC can attain those numbers as well.

Interestingly, the fastest CPU based network switches tend to do full kernel bypass. The kernel is generally slow compared to OVS and VPP, especially when they traverse over something like DPDK.

Kernel bypass in DPDK grants the application direct access to DMA buffers so that the kernel is no longer involved. This is not because the kernel is slow, but because many small syscalls are expensive and putting your entire app in the kernel is a bad idea.

There is no kernel bypass in wireguard-go, just a user-space implementation fast implementation with smart use of syscalls to minimize the overhead of being split between user-space and kernel-space.

With io_uring, DPDK-style kernel bypass might stop making sense altogether.

Re: Surpassing 10Gb/S over Tailscale

#53
post #39
post #22

The missing feature from Tailscale for me is the ability to host a Tailscale only DNS zone. They have Magic DNS, but that only works for individual Tailscale nodes. I want multiple DNA records pointing to a single Tailscale node. Would be even better if I could use my own domain (subdomain even better) instead of their long `foo-bar.ts.net` domain. Currently need to do this manually, but seems overly redundant since…

Step 1: install Tailscale and Docker on a VM or whatever Step 2: set up a Technitium container in host networking mode Step 3: configure Technitium with a stub zone pointing your ts.net name at 100.100.100.100 Step 4: set up a zone for whatever.tld Step 5: set up a DNAME record for ts.whatever.tld pointing at your ts.net domain Result: querying this new DNS server with machine.ts.whatever.tld resolves to machine.blah…

I know this can be done manually (and I do), but the issue with that is that: 1. It's manual 2. Single point of failure of this server that was needed

My point was that MagicDNS is implemented in the Tailscale client on each machine (fault tolerant, 0ms latency) and has almost all the things necessary (DNS resolver, push mechanism for record updates) except for a custom defined zone.

Running `drill @100.100.100.100 ..ts.net` is 0ms because it's local, and doesn't depend on a single DNS server running somewhere on my Tailscale network.

Re: Surpassing 10Gb/S over Tailscale

#54
post #40

Earlier quoted context omitted.

go is pretty fast in fact, i have a standing bet with some of my rustacean friends that they can't show me a typical HTTP service in rust, which has performance numbers (rps, latency, throughput) that i can't meet or beat in go of course lots of caveats there, what does normal-ish mean, well probably most of the work is gonna be i/o bound, it should run on normal server-class hardware, et cetera et cetera but nothing…

For most compiled languages or languages with very good VMs like Java benchmarks are really testing the quality of the implementation and the depth of the implementor's understanding. I'd bet that very good Go and Rust programmers could probably converge to almost identical performance. What I wouldn't be on is that Go could equal Rust in the area of small memory footprint or on small devices.

> What I wouldn't be on is that Go could equal Rust in the area of small memory footprint or on small devices.

I haven't found a microcontroller that's too small for tinygo. I have even used time.Format(time.RFC3339) on one before. $1 spent on a microcontroller is the ultimate luxury these days.

Re: Surpassing 10Gb/S over Tailscale

#55
post #12

Tailscale is awesome, so damn recommended. Taildrop (AirDrop for everything, included in Tailscale) is especially recommended, it makes it so damn easy to send files between all your devices.

https://tailscale.com/kb/1106/taildrop/ seems to be the docs. It's the first I hear of this. I wonder if there's any big advantage of this for someone who is already using syncthing for the same purpose? Biggest thing I could hope for is that it's faster. But I generally don't keep Tailscale running on mobile because I don't need it to and don't like the persistent notification.

Syncthing is slower, you need to act on both devices.

With Taildrop you just need to share something with a couple of clicks, and it'll appear on the device(s) you share it to.

Re: Surpassing 10Gb/S over Tailscale

#56
post #53
post #39

Earlier quoted context omitted.

Step 1: install Tailscale and Docker on a VM or whatever Step 2: set up a Technitium container in host networking mode Step 3: configure Technitium with a stub zone pointing your ts.net name at 100.100.100.100 Step 4: set up a zone for whatever.tld Step 5: set up a DNAME record for ts.whatever.tld pointing at your ts.net domain Result: querying this new DNS server with machine.ts.whatever.tld resolves to machine.blah…

I know this can be done manually (and I do), but the issue with that is that: 1. It's manual 2. Single point of failure of this server that was needed My point was that MagicDNS is implemented in the Tailscale client on each machine (fault tolerant, 0ms latency) and has almost all the things necessary (DNS resolver, push mechanism for record updates) except for a custom defined zone. Running `drill @100.100.100.100 .…

Yep, that's fair. I actually run this setup on every machine in my lab. Technitium is so light weight and with this setup I don't need to jump through any hoops to get Docker containers to resolve Tailscale names.

Re: Surpassing 10Gb/S over Tailscale

#57
post #56
post #53

Earlier quoted context omitted.

I know this can be done manually (and I do), but the issue with that is that: 1. It's manual 2. Single point of failure of this server that was needed My point was that MagicDNS is implemented in the Tailscale client on each machine (fault tolerant, 0ms latency) and has almost all the things necessary (DNS resolver, push mechanism for record updates) except for a custom defined zone. Running `drill @100.100.100.100 .…

Yep, that's fair. I actually run this setup on every machine in my lab. Technitium is so light weight and with this setup I don't need to jump through any hoops to get Docker containers to resolve Tailscale names.

I'd never heard of Technitium, but was intrigued looking at. Was thinking "hmmm what could I do with this" and then had to refrain from creating another project just because.

TBH I find Docker networking a struggle and usually disable the `iptables` stuff and end up configuring my own rules. Painful, but at least less intrusive.

On the note of Tailscale+Docker networking, gluetun[0] is pretty awesome. It runs a Wireguard (not tailscale compatible, yet) instance within a Docker container and then you share that networking namespace with the other containers effectively confining them to the VPN. Comes with basic container namespace firewall configuration and DNS over TLS configuration.

[0] https://github.com/qdm12/gluetun

Re: Surpassing 10Gb/S over Tailscale

#58
post #31

Pretty amazing that you can achieve such a throughput in a Golang userspace program. I wonder if other UDP based protocols like QUIC can attain those numbers as well.

Interestingly, the fastest CPU based network switches tend to do full kernel bypass. The kernel is generally slow compared to OVS and VPP, especially when they traverse over something like DPDK.

Are there consumer (<$2k) network switches that can do Wireguard in a very fast path?

Re: Surpassing 10Gb/S over Tailscale

#59
post #31

Pretty amazing that you can achieve such a throughput in a Golang userspace program. I wonder if other UDP based protocols like QUIC can attain those numbers as well.

Interestingly, the fastest CPU based network switches tend to do full kernel bypass. The kernel is generally slow compared to OVS and VPP, especially when they traverse over something like DPDK.

It's not "kernel is slow", kernel when left to its own devices is plenty fast, the reason is that when you want to make decision about packet in userspace (vs telling kernel what to do with it via various interfaces) that kernel logic would just be overhead.

It's similar for applications; if you can, say, decode whole DNS packet in one go, you don't really want kernel to spend time decoding UDP packet, then you decoding the rest of the packet; doing it in one step is much faster.

Re: Surpassing 10Gb/S over Tailscale

#60
post #40

Earlier quoted context omitted.

go is pretty fast in fact, i have a standing bet with some of my rustacean friends that they can't show me a typical HTTP service in rust, which has performance numbers (rps, latency, throughput) that i can't meet or beat in go of course lots of caveats there, what does normal-ish mean, well probably most of the work is gonna be i/o bound, it should run on normal server-class hardware, et cetera et cetera but nothing…

For most compiled languages or languages with very good VMs like Java benchmarks are really testing the quality of the implementation and the depth of the implementor's understanding. I'd bet that very good Go and Rust programmers could probably converge to almost identical performance. What I wouldn't be on is that Go could equal Rust in the area of small memory footprint or on small devices.

> I'd bet that very good Go and Rust programmers could probably converge to almost identical performance.

I'd imagine probably not purely because Rust uses LLVM which is VERY good at optimizing, while Go compiler is simpler and made for speed of compilation first. If Go got LLVM frontend yeah, maybe

> What I wouldn't be on is that Go could equal Rust in the area of small memory footprint or on small devices.

Well, Go is GCed, that automatically makes it use at least a bit more, and also carrying code for GC with each program.

Post reply on HN