Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We improved the performance of a userspace TCP stack in Go

coder.com

Post reply on HN
Print view View original

81–90 of 133 posts

Re: We improved the performance of a userspace TCP stack in Go

#81

Post by weitendorf »

parhamn wrote: post #74
tptacek wrote: post #71

Earlier quoted context omitted.

No. The providers that did so soundly used virtualization to accomplish this, and a big part of the appeal of K8s is having a much lightweight unit of scheduling than full virtualization. gVisor is a middle ground between full virtualization and shared-kernel multitenant (which has an abysmal security track record).

Virtualization, lxc, containers (and K8s), etc were solutions to "secure shared environments". And they have an order of magnitude lower performance hit than gvisor does (Google 'cloudrun python startup times' if you're curious on the real impact of this stuff). Have we proven they're not secure and safe? Have we broken out of containers yet? Heroku was running LXC for years before docker, did they run into major sec…

Containers are not compute environments, their runtimes are, and gvisor (runsc) is one implementation of that. Docker engine (~runc) is another. It has similar performance characteristics to gvisor afaict looking online (the minimum cold start times I'm seeing are 500ms which I've beat in gvisor), yet implements less security features.

If by virtualization you mean VMs, gvisor can be more performant than those based on my experience. For example, AWS claims a p0 coldstart time of ~500ms using Firecracker but I know firsthand that applications sandboxed by gvisor can be made to cold start in significantly less time (like less than half): https://catalog.workshops.aws/java-on-aws-lambda/en-US/03-sn..., and you should be able to confirm this yourself by using products that leverage Gvisor under the hood or with your own testing. I actually worked on this (using gvisor, but working on adjacent tech) for years...

> Have we broken out of containers yet?

Sure, how about https://scout.docker.com/vulnerabilities/id/CVE-2024-21626 where runc (Docker) exposed the host filesystem to containerized applications? Precisely the kind of exploit gvisor is designed to prevent.

I'll note that a lot of people are thinking about how to reduce sandbox overhead in multitenant PaaS and it's one of the things I want to eventually address in my own startup. But I think blindly hating on gvisor because of a nebulous dislike of overhead really is misplaced without considering its alternatives.

Re: We improved the performance of a userspace TCP stack in Go

#83

Post by parhamn »

weitendorf wrote: post #81
parhamn wrote: post #74

Earlier quoted context omitted.

Virtualization, lxc, containers (and K8s), etc were solutions to "secure shared environments". And they have an order of magnitude lower performance hit than gvisor does (Google 'cloudrun python startup times' if you're curious on the real impact of this stuff). Have we proven they're not secure and safe? Have we broken out of containers yet? Heroku was running LXC for years before docker, did they run into major sec…

Containers are not compute environments, their runtimes are, and gvisor (runsc) is one implementation of that. Docker engine (~runc) is another. It has similar performance characteristics to gvisor afaict looking online (the minimum cold start times I'm seeing are 500ms which I've beat in gvisor), yet implements less security features. If by virtualization you mean VMs, gvisor can be more performant than those based…

The charts you linked in the performance guide show a 30x syscall overhead in runsc vs runc (careful quit a few of the charts are logrithmic). That's insane! They also go on claim a 20% tensorflow workload difference.

> I want to eventually address in my own startup.

You worked on CloudRun and their performance is dogshit. Seriously google it theres like 100 stack overflow questions on the subject. It's common enough a query Google even suggests follow up questions like: "Why is cloud run so slow?".

Now your answer might be "avoid syscalls", "don't do anything on the file system (oh by the way your file system is memory mapped hehe)", "interpreters can be slow to load their code, sorry", "look at these charts its not as bad as you say", "tcp overhead is only 30%", etc but your next set of customers wont have the same vendor lock in you enjoyed at Google.

Then do the same query for "Digital Ocean Apps slow", also gvisor. And bam you'll have a long list of customers ready to use your better version! Perhaps Google and Digital Ocean will enlist your expertise (again).

Re: We improved the performance of a userspace TCP stack in Go

#84

Post by taeric »

tptacek wrote: post #69
Xelynega wrote: post #67

Earlier quoted context omitted.

I don't think you understand, and based on your reply it doesn't sound like I'm mistaken. With this CLI I am able to listen for external packets to port 80 from userspace without any elevated permissions and intercept traffic that's going to an application that's bound to that port on the OS? Edit: I think I understand what you're trying to do, but if I do then traffic is going from the kernel UDP stack to the userla…

I'm not sure I can make this any simpler for you or easier to get your head around. If it helps: the idea here is giving an invocation of your program its own IP address . It can then do whatever it likes with TCP/IP for that address; its own routing, arbitrary protocols, whatever. The Go standard library makes it extremely easy to integrate. To the OS, it's all just ordinary socket code.

I think it is a valid question on why not use unprivileged ports, though? Or am I also missing something?

Re: We improved the performance of a userspace TCP stack in Go

#85

Post by parhamn »

weitendorf wrote: post #75
parhamn wrote: post #33

Earlier quoted context omitted.

In the issue of abysmal performance on cloud-compute/PaaS Im talking about the container runtime (most Paas is gVisor or Firecracker, no?) cloudrun, DO, modal, etc. But given this article is about improving gvisors userland tcp performance significantly, it seems like the netstack stuff causes major performance losses too. I saw a github link in another top article today https://github.com/misprit7/computerraria wher…

In the context of coder, the userspace TCP overhead should be negligible. Based on https://gvisor.dev/docs/architecture_guide/performance/ and assuming runc is mostly just using the regular kernel networking stack (I think it does, since it mostly just does syscall filtering?) it should be at most a 30% direct TCP performance hit. But in a real application you typically only spend a negligible amount of total time in…

TCP overhead is part of the story. Theres 20-40x overhead in syscalls, 20% running a tensorflow project end to end, 50% fewer RPS in redis, etc.

Re: We improved the performance of a userspace TCP stack in Go

#86

Post by parhamn »

weitendorf wrote: post #39
parhamn wrote: post #19

I don't know anything about Coder, but Gvisor proliferation is annoying. It's a boon for cloud providers, helping them find another way to get a large multiple performance decrease per dollar spent in exchange for questionable security benefits. And I'm seeing it everywhere now.

I don't understand - what do you suggest as an alternative to Gvisor? > large multiple performance decrease per dollar spent Gvisor helps you offer multi-tenant products which can be actually much cheaper to operate and offer to customers, especially when their usage is lower than a single VM would require. Also, a lot of applications won't see big performance hits from running under Gvisor depending on their resourc…

> I don't understand - what do you suggest as an alternative to Gvisor?

Their performance documents you linked claim vs runc: 20-40x syscall overhead, half of redis' QPS, and a 20% increase in runtime in a sample tenserflow script. Also google "CloudRun slow" and "Digital Ocean Apps slow", both are Gvisor.

Literally anything else.

Re: We improved the performance of a userspace TCP stack in Go

#87

Post by tptacek »

parhamn wrote: post #85
weitendorf wrote: post #75

Earlier quoted context omitted.

In the context of coder, the userspace TCP overhead should be negligible. Based on https://gvisor.dev/docs/architecture_guide/performance/ and assuming runc is mostly just using the regular kernel networking stack (I think it does, since it mostly just does syscall filtering?) it should be at most a 30% direct TCP performance hit. But in a real application you typically only spend a negligible amount of total time in…

TCP overhead is part of the story. Theres 20-40x overhead in syscalls, 20% running a tensorflow project end to end, 50% fewer RPS in redis, etc.

We are still talking about people using runsc/runc. That's not what `coder` is doing. All they did was poach a (popular) networking library from the gVisor codebase. None of this benchmarking has anything to do with their product.

Re: We improved the performance of a userspace TCP stack in Go

#88

Post by tptacek »

taeric wrote: post #84
tptacek wrote: post #69

Earlier quoted context omitted.

I'm not sure I can make this any simpler for you or easier to get your head around. If it helps: the idea here is giving an invocation of your program its own IP address . It can then do whatever it likes with TCP/IP for that address; its own routing, arbitrary protocols, whatever. The Go standard library makes it extremely easy to integrate. To the OS, it's all just ordinary socket code.

I think it is a valid question on why not use unprivileged ports, though? Or am I also missing something?

"Unprivileged ports" is just a case in point for everything you would need privileges to do, from binding arbitrary ports to adding arbitrary addresses and running local servers on them, etc, etc, etc. The point is: turning "complicated" network features that require privilege into simple unprivileged socket code.

Re: We improved the performance of a userspace TCP stack in Go

#89

Post by amscanne »

parhamn wrote: post #86
weitendorf wrote: post #39

Earlier quoted context omitted.

I don't understand - what do you suggest as an alternative to Gvisor? > large multiple performance decrease per dollar spent Gvisor helps you offer multi-tenant products which can be actually much cheaper to operate and offer to customers, especially when their usage is lower than a single VM would require. Also, a lot of applications won't see big performance hits from running under Gvisor depending on their resourc…

> I don't understand - what do you suggest as an alternative to Gvisor? Their performance documents you linked claim vs runc: 20-40x syscall overhead, half of redis' QPS, and a 20% increase in runtime in a sample tenserflow script. Also google "CloudRun slow" and "Digital Ocean Apps slow", both are Gvisor. Literally anything else.

A decent while ago, I was the original author of that performance guide. I tried to lay out the set of performance trade-offs in an objective and realistic way. It is shocking to me that you’re spending so much time commenting on a few figures from there, ostensibly w/o reading it.

System call overhead does matter, but it’s not the ultimate measure of anything. If it were, gVisor with the KVM platform would be faster than native containers (looking at the runsc-kvm data point which you’ve ignored for an unknown reason). But it is obviously more complex than that alone. For example, let’s click down and ask — how is it even possible to be faster? The default docker seccomp profile itself installs an eBPF filter that slows system calls by 20x! (And this path does not apply within the guest context.) On that basis, should you start shouting that everyone should stop using Docker because of the system call overhead? I would hope not, because looking at any one figure in isolation is dumb — consider the overall application and architecture. Containers themselves have a cost (higher context switch time due to cgroup accounting, costs to devirtualize namespaces in many system calls, etc.) but it’s obviously worth it in most cases.

The redis case is called out as a worst case — the application itself does very little beyond dispatching I/O, so almost everything manifests as overhead. But if you’re doing something that has 20% overhead, you need hard security boundaries, and fine-grained multi-tenancy can lower costs by 80% it might make perfect sense. If something doesn’t work for you because your trade-offs are different, just don’t use it!

Re: We improved the performance of a userspace TCP stack in Go

#90

Post by zxt_tzx »

I met one of the founders of Coder.com, he's a really cool dude. It's a pity that it is a product aimed more at enterprises than individual developers, else it would have far more developer mindshare.

Unlike, say, GitHub Codespaces, running something like this on your own infra means your incentives and Coder.com's are aligned, i.e. both of you want to reduce your cloud costs (as opposed to, say, GitHub running on Azure gives them an opportunity and incentive to mark up on Azure cloud costs).

Post reply on HN