Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We improved the performance of a userspace TCP stack in Go

coder.com

Post reply on HN
Print view View original

51–60 of 133 posts

Re: We improved the performance of a userspace TCP stack in Go

#51

Post by anyfoo »

Xelynega wrote: post #49
tptacek wrote: post #27

Earlier quoted context omitted.

I understand the impulse, but I think it misconstrues the "red tape" this method avoids. It's sidestepping a quirky OS limitation, which dates back to an era of "privileged ports" and multi-user machines. It's not really sidestepping any sort of modern policy boundary. For instance: you could do the exact same thing with WebSockets (and people do).

You can also sidestep that "quirky OS limitation" by just setting the first unprivileged port to 0(ip_unprivileged_port_start), no need for an new stack. https://www.kernel.org/doc/Documentation/networking/ip-sysct...

Getting that change onto the system sounds like "at best a big delay and at worst a nonstarter".

Re: We improved the performance of a userspace TCP stack in Go

#52

Post by raggi »

iangudger wrote: post #31
dpeckett wrote: post #20

Really cool to see others hacking on netstack, bit of a shame it's tied up in the gVisor monorepo (and all the Bazel idiosyncracies) but it's a very neat piece of kit. I've actually been hacking on a similar FOSS project lately, with a focus on building what I'm calling a layer 3 service mesh for the edge. More or less came out of my learned hatred for managing mTLS at scale and my dislike for shoving everything thro…

If you want to use netstack without Bazel, just use the go branch: https://github.com/google/gvisor/tree/go go get gvisor.dev/gvisor/pkg/tcpip@go The go branch is auto generated with all of the generated code checked in.

hey Ian, long time. Is there any chance y'all could swap out main so that main contains the generated code version?

I don't know the status on those export tools these days as I left the company years ago, but if they could sync with a different branch.

This would help various folks quite a bit, as for example tsnet users often fall into the trap of trying to do `go get -u`, which then pulls a non-functional gvisor version.

Re: We improved the performance of a userspace TCP stack in Go

#53

Post by anyfoo »

wmf wrote: post #2

"Asking for elevated permissions inside secure clusters at regulated financial enterprises or top secret government networks is at best a big delay and at worst a nonstarter." But exfiltrating data with a userspace VPN is totally fine? I'm also wondering why not use TLS.

You can't control what information flows through an outbound connection, not even in trivial cases. Even if you straight go ahead and say "I allow you to make this connection, but I'm not even allowing you to send any data", you have timing sidechannels to deal with. In any more reasonable case, an almost infinite number of things can be used to exfiltrate any data you want, even if you think you have not only full application-level inspection, but even application-level rewrite.

Pretty much the only thing you can do is somewhat filter out known-bad, not directly motivated outbound traffic, such as malware payloads with very clear signatures. This only works if it's "not directly motivated", because as soon as there's a person who wants to do it, they can skirt around it again.

Re: We improved the performance of a userspace TCP stack in Go

#54

Post by raggi »

wmf wrote: post #2

"Asking for elevated permissions inside secure clusters at regulated financial enterprises or top secret government networks is at best a big delay and at worst a nonstarter." But exfiltrating data with a userspace VPN is totally fine? I'm also wondering why not use TLS.

fwiw, you technically don't need a privileged container to use tun, you just need suitable permissions on the kernel tun interfaces.

Re: We improved the performance of a userspace TCP stack in Go

#55

Post by Xelynega »

vlovich123 wrote: post #4
tazjin wrote: post #3

Earlier quoted context omitted.

Yeah, the optimisations are cool of course, but (maybe due to being unfamiliar with the tool?!) I didn't understand why they can't just `listen(2)`.

It’s answered in the opening paragraph although I’ll admit I’m still unclear. > We are committed to keeping your data safe through end-to-end encryption and to making Coder easy to run across a wide variety of systems from client laptops and desktops to VMs, containers, and bare metal. If we used the TCP implementation in the OS, we’d need a way for the TCP packets to get from the operating system back into Coder for…

I'm confused on why they would need a TUN device for a client or server application, so why they would need this solution in the first place(even with their explanation).

As I understand the only reason you'd use a TUN interface is if you want to send/receive raw IP packets. Their marketing doesn't make it very clear what their product does, but I can't see a reason it would need to send/receive raw IP packets rather than TCP/UDP packets over a specific port...

Re: We improved the performance of a userspace TCP stack in Go

#56

Post by raggi »

syzcowboy99 wrote: post #14
pantalaimon wrote: post #10

The obvious question is: How does it compare to the in-Kernel TCP stack?

gVisor's netstack is still much slower than the kernel's (and likely always will be). The goal of this userspace netstack is not to compete with the kernel on performance, but offer an alternative that is more portable and secure.

for some definition of portable which is deeply tied to the go runtime

Re: We improved the performance of a userspace TCP stack in Go

#57

Post by nonameiguess »

Xelynega wrote: post #49
tptacek wrote: post #27

Earlier quoted context omitted.

I understand the impulse, but I think it misconstrues the "red tape" this method avoids. It's sidestepping a quirky OS limitation, which dates back to an era of "privileged ports" and multi-user machines. It's not really sidestepping any sort of modern policy boundary. For instance: you could do the exact same thing with WebSockets (and people do).

You can also sidestep that "quirky OS limitation" by just setting the first unprivileged port to 0(ip_unprivileged_port_start), no need for an new stack. https://www.kernel.org/doc/Documentation/networking/ip-sysct...

When they're talking about classified defense networks, the actual restrictions they mean is least privilege and separation of duties. Devs are not admins. They don't get root privilege on their machines. They can't create virtual network interfaces and they also can't change kernel settings. But if you put a full TCP/IP stack in userspace, well, they can run that and do whatever they want with it.

To answer the upstream question about why arbitary outbound connections are allowed, they're not. This is connecting to a cloud development environment, and I would have to assume this service can be self-hosted, because on a classified network, the "cloud" isn't the cloud as Hacker News readers know it. Amazon et all run private data centers on US military installations that only the military and the IC can access and they're airgapped from the Internet. If you're on a workstation that can access this environment, that's all it can access. The only place you can exfiltrate data to is other military-controlled servers.

Re: We improved the performance of a userspace TCP stack in Go

#58

Post by parhamn »

tptacek wrote: post #48
parhamn wrote: post #46

Earlier quoted context omitted.

> The netstack stuff here has nothing to do with the rest of gVisor. How so? Besides being part of it, it is at least similar in the group of "bloated slow userland implementation of things the kernel handles well"

A TCP/IP stack is not an "implementation of syscalls". The things most netstack users do with netstack have nothing to do with wanting to move the kernel into userland and everything to do with the fact that the kernel features they want to access are either privileged or (in a lot of IP routing cases) not available at all. Netstack (like any user-mode IP stack) allows programs to do things they couldn't otherwise do…

> The gVisor/perf thing is a tendentious argument

Interesting to dismiss it as such. The gvisor netstack is a (big) part of gvisor and this article is discussing how the performance of that component was, and could well still be, garbage.

These tools bring marginal capability and performance gains, shoved down peoples throat by manufacturing security paranoia. Oh an it all happens to cost you like 10x time, but look at the shiny capabilities, trust me it couldn't be done before! A netsec and infra peddlers wet dream.

Re: We improved the performance of a userspace TCP stack in Go

#59

Post by Xelynega »

syzcowboy99 wrote: post #14
pantalaimon wrote: post #10

The obvious question is: How does it compare to the in-Kernel TCP stack?

gVisor's netstack is still much slower than the kernel's (and likely always will be). The goal of this userspace netstack is not to compete with the kernel on performance, but offer an alternative that is more portable and secure.

How is it more portable or secure than an API that's been stable for decades, and getting constant security fixes?

I see an explanation in their blog about avoiding TUN devices since they require elevated permissions, but why would you need a TUN device to send data to/from an application? I can't understand what their product does from the marketing material but it doesn't look like it would require constructing raw IP packets instead of TCP/UDP packets and letting the OS wrap them in the other layers.

Re: We improved the performance of a userspace TCP stack in Go

#60

Post by raggi »

It's great to see this, I know the team went on a long journey through this and the blog makes it almost look shorter and simpler than it was. I'm hoping one day we can all integrate the support for GSO that's been landing in gvisor too, but so far we've (tailscale) not had a chance to look deeply into that yet. It was really effective for our tun and UDP interfaces though.
Post reply on HN