Live data from Hacker News

Surpassing 10Gb/S over Tailscale

tailscale.com

21–30 of 84 posts

Re: Surpassing 10Gb/S over Tailscale

#21

Pretty amazing that you can achieve such a throughput in a Golang userspace program. I wonder if other UDP based protocols like QUIC can attain those numbers as well.

is the go userspace program actually shoveling this data or are they in-kernel buffer copies a la sendfile and the like

Re: Surpassing 10Gb/S over Tailscale

#22
The missing feature from Tailscale for me is the ability to host a Tailscale only DNS zone.

They have Magic DNS, but that only works for individual Tailscale nodes. I want multiple DNA records pointing to a single Tailscale node. Would be even better if I could use my own domain (subdomain even better) instead of their long `foo-bar.ts.net` domain.

Currently need to do this manually, but seems overly redundant since Tailscale already does 90% of this with MagicDNS and is fast because it's in their client vs a remote server.

Re: Surpassing 10Gb/S over Tailscale

#23

Pretty amazing that you can achieve such a throughput in a Golang userspace program. I wonder if other UDP based protocols like QUIC can attain those numbers as well.

is the go userspace program actually shoveling this data or are they in-kernel buffer copies a la sendfile and the like

Yes, userspace has to touch the data to encrypt/decrypt it.

Re: Surpassing 10Gb/S over Tailscale

#24
post #11

Earlier quoted context omitted.

I've switched to tailscale because their nat busting is actually hard to do "by hand"

That’s sort of the problem, right. Joining two networks is pretty simple, once you do it a few times. I remember when it was mandatory to know how to set up an email server (for more than one user), configure a secure FTP (+ WebDAV for a little while), and probably other things I’ve totally forgetting about. These things were passed down from senior to junior like we pass down how to write Docker images, and set up o…

none of those things are analogous to Tailscale having done loads of hard work to automate NAT busting.

Re: Surpassing 10Gb/S over Tailscale

#25
post #14

Nice improvements! I'd be interested to see how much overhead tailscales magicsock adds and what a flamegraph after the change looks like. Mostly crypto or still a lot of networking syscall time?

magicsock definitely does a bunch more work, and we do look at both profiles. The magicsock profile is harder to read as a consequence of being a more complex path, adding packet filters, the indirection for DERP and other NAT busting details, etc. Jordan did do some optimizations in the magicsock path alongside this wireguard-go work to get us over the 10gbps line.

Overall the summary of time spent is still a similar story at the coarse scale - our recent optimizations mean that we're getting ever closer to the point where we need to start working on the next layer, such as optimizing the queues (visible here in the chanrecv and scheduler times - Go runtime stuff), and once we get that out of the way things like crypto and copying will become targets. The work goes on, we have lots of plans and ideas!

Re: Surpassing 10Gb/S over Tailscale

#26
post #11

Earlier quoted context omitted.

I've switched to tailscale because their nat busting is actually hard to do "by hand"

That’s sort of the problem, right. Joining two networks is pretty simple, once you do it a few times. I remember when it was mandatory to know how to set up an email server (for more than one user), configure a secure FTP (+ WebDAV for a little while), and probably other things I’ve totally forgetting about. These things were passed down from senior to junior like we pass down how to write Docker images, and set up o…

Tailscale has several competitors such as ZeroTier and Nebula. There does appear to be a winner-take-all dynamic where being slightly better lets Tailscale take 10x more mindshare than competitors, but I don't see any way around that.

Re: Surpassing 10Gb/S over Tailscale

#28
post #12

Tailscale is awesome, so damn recommended. Taildrop (AirDrop for everything, included in Tailscale) is especially recommended, it makes it so damn easy to send files between all your devices.

https://tailscale.com/kb/1106/taildrop/ seems to be the docs. It's the first I hear of this. I wonder if there's any big advantage of this for someone who is already using syncthing for the same purpose? Biggest thing I could hope for is that it's faster. But I generally don't keep Tailscale running on mobile because I don't need it to and don't like the persistent notification.

Sync, continuous backup and transfer are all quite different use-cases.

Most backup/sync products are designed to work in the background and often require upload before download. I don’t know if syncthing does streaming syncs though.

Another difference is transfers can easily be untrusted, as in sender and receiver don’t need access to each others file systems. Take magic wormhole (or email attachments for that matter) as an example.

Taildrop is somewhere in between – I think you have to be on same tail net, but no need for awareness of the other device’s file system.

Re: Surpassing 10Gb/S over Tailscale

#29
What‘s missing from all these figure is the resulting latency. It‘s often the case that vendors show impressive throughput numbers, but then the latency is terrible at that throughput.

Do you have those numbers as well?

Re: Surpassing 10Gb/S over Tailscale

#30
post #20

Earlier quoted context omitted.

I mean, setting up a WireGuard vpn is pretty darn simple, even into a k8s cluster. It’s not rocket science or anything; which is kinda my point. They make it too easy, and that worries me.

Setting up a few p2p wg VPNs is manageable. However, when you have 10 nodes and need to add one more node, you now need to update all other nodes so they can speak p2p. Management with scale is the struggle.

If you have 10 nodes, you should already be automating with ansible/chef/puppet/whatever, at which point adding another link config is easy.
Post reply on HN