Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Surpassing 10Gb/S over Tailscale

tailscale.com

Post reply on HN
Print view View original

71–80 of 84 posts

Re: Surpassing 10Gb/S over Tailscale

#71

Post by evulhotdog »

hardwaresofton wrote: post #70
psanford wrote: post #27

Earlier quoted context omitted.

Nebula predates tailscale.

link for the lazy: https://github.com/slackhq/nebula Also with no ill intent looks like tailscale has the far more effective marketing organization :)

The team that built nebula at Slack actually split off and are building a similar type of network, that they give you a single pane of glass to manage, so you don’t have to manage a bunch of separate lighthouse devices manually.

I evaluated it when going through the many available options out there a year or two ago, and it was still pretty green feature wise. Still very cool and could work very well depending on your use cases. See: https://www.defined.net/

Re: Surpassing 10Gb/S over Tailscale

#73

Post by renewiltord »

wmf wrote: post #61
renewiltord wrote: post #58

Earlier quoted context omitted.

Are there consumer (<$2k) network switches that can do Wireguard in a very fast path?

By their nature as L2/L3 devices, I wouldn't expect switches to ever support Wireguard. I also haven't heard of any hardware Wireguard yet. The fastest implementation so far might be TNSR which just squeaks in under $2,000.

Right, I misspoke. I should know better. I intended to say "a network device".

Re: Surpassing 10Gb/S over Tailscale

#74

Post by [deleted] »

_0w8t wrote: post #37
preseinger wrote: post #35

Earlier quoted context omitted.

go is pretty fast in fact, i have a standing bet with some of my rustacean friends that they can't show me a typical HTTP service in rust, which has performance numbers (rps, latency, throughput) that i can't meet or beat in go of course lots of caveats there, what does normal-ish mean, well probably most of the work is gonna be i/o bound, it should run on normal server-class hardware, et cetera et cetera but nothing…

Caddy (a web server written in Go) is like two times slower than Nginx on many benchmarks.

[deleted]

Re: Surpassing 10Gb/S over Tailscale

#76

Post by rektide »

Userspace networking makes me a bit sad in that it's much harder for users to observe or instrument. It's convenient for app developers, but to lock users out of seeing what is happening on their own systems feels awful.

Re: Surpassing 10Gb/S over Tailscale

#77

Post by madjam002 »

2bluesc wrote: post #22

The missing feature from Tailscale for me is the ability to host a Tailscale only DNS zone. They have Magic DNS, but that only works for individual Tailscale nodes. I want multiple DNA records pointing to a single Tailscale node. Would be even better if I could use my own domain (subdomain even better) instead of their long `foo-bar.ts.net` domain. Currently need to do this manually, but seems overly redundant since…

There is an open GitHub issue for this and it’s already been implemented in the Tailscale client, it’s really nice too as the DNS records are pushed out to the local DNS resolver on each Tailscale client, rather than being lookups to a separate server, so it’s super fast.

Unfortunately there aren’t any options for it on the Tailscale control panel, but if you use Headscale you can configure it and take advantage of it now.

Re: Surpassing 10Gb/S over Tailscale

#78

Post by ThePhysicist »

preseinger wrote: post #35
ThePhysicist wrote: post #6

Pretty amazing that you can achieve such a throughput in a Golang userspace program. I wonder if other UDP based protocols like QUIC can attain those numbers as well.

go is pretty fast in fact, i have a standing bet with some of my rustacean friends that they can't show me a typical HTTP service in rust, which has performance numbers (rps, latency, throughput) that i can't meet or beat in go of course lots of caveats there, what does normal-ish mean, well probably most of the work is gonna be i/o bound, it should run on normal server-class hardware, et cetera et cetera but nothing…

That's my experience as well. Recently I rewrote a Golang-based QUIC server in Rust and I had a hard time getting it to perform equally well. Certainly possible but requires a lot of hand-tuning and knowing exactly what you do. In Golang you just spawn a Go routine for each request and avoid lock-based shared state as much as possible and you're mostly good, the runtime will manage all aspects like number of threads, allocations etc. for you.

One area where Rust is still better are memory-constrained environments e.g. on mobile and on microcontroller, though there's tinygo and the Go runtime gets slimmer as well, so now you can have binaries and memory footprints smaller than 5 MB on most mobile platforms, which is absolutely acceptable even for budget phones. I think Tailscale e.g. runs their modified version of wireguard-go on all mobile clients without issues.

Re: Surpassing 10Gb/S over Tailscale

#79

Post by ThePhysicist »

adam_arthur wrote: post #36
preseinger wrote: post #35

Earlier quoted context omitted.

go is pretty fast in fact, i have a standing bet with some of my rustacean friends that they can't show me a typical HTTP service in rust, which has performance numbers (rps, latency, throughput) that i can't meet or beat in go of course lots of caveats there, what does normal-ish mean, well probably most of the work is gonna be i/o bound, it should run on normal server-class hardware, et cetera et cetera but nothing…

Seems Rust places well in some composite benchmarks. Go is further down the list. Of course this depends on the quality of the implementation and doesn't account for UX/usability https://www.techempower.com/benchmarks/#section=data-r21&tes...

According to that benchmark, Javascript seems to be the way to go.

Re: Surpassing 10Gb/S over Tailscale

#80

Post by majke »

Okay, as far as I understand this writeup.

There are two sides, userspace UDP socket to receive wg packets on. Then the tap file descriptor to receive unencrytped packets from the host OS.

To speed up the userspace UDP socket it's desirable to use UDP_GRO flag on RX, and UDP_SEGMENT flag on TX. `tx-udp-segmentation` is a HW help for the latter. No need for any checksums and stuff. This is just speedup for userspace "classic" UDP socket.

However, buffering with UDP_GRO is interesting, since you need to pass potentially large 64KiB buffer to kernel since you don't know how large the next GRO-packet is. (this is a digression)

On the tap side, the article implies they enabled TUN_F_TSO4, which is a magical offload flag on tun interface. With it it is possible to get large pakets form the host OS. This is where it gets interesting. If you get a very large block from the host, like say 14KiB or larger.... how do you push it to the wireguard socket? I guess it's nececesary to packetize it back to small-MSS packets before encrypting. That means recreating TCP headers (with seq numers) and filling the checksum. This sounds like "fun".

The same on TX side towards the host... if you get a number of TCP segments from the wg tunnel, decrypt them.... do you push them as one large TUN_F_TSO segment to tun? or do you push one-by-one and rely on the kernel to GRO them? I didn't quite get it from the article. Or maybe it's possible to send large packets over wg without segmentation?

The same discussion is about UDP. With UDP you can use TUN_F_USO, however, this is only available in kernel 6.2. This might be why there arent' too many UDP numbers in the article.

Post reply on HN