Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSH and User-Mode IP WireGuard

fly.io

Post reply on HN
Print view View original

41–50 of 104 posts

Re: SSH and User-Mode IP WireGuard

#41

Post by kasey_junk »

londons_explore wrote: post #39
tptacek wrote: post #28

Earlier quoted context omitted.

You need WireGuard to SSH to machines at Fly (that's a good thing). You don't have WireGuard installed on a particular machine. That's OK, because there's a portable, userland, Golang implementation of not only WireGuard but all of TCP/IP that can be imported into any Go program. Go programs can BYO network stacks. That's crazy. The end.

Still seems like a downgrade for actual users... I just want to be able to type ssh instance7.service.zone.user.fly.io into my console, and be connected... I don't actually care about compiling my own custom ssh client written in go, however neat its implementation might be...

But! They shipped that in their go client program so you don’t have to.

Re: SSH and User-Mode IP WireGuard

#42

Post by ignoramous »

tptacek wrote: post #2

I added some example code to the post, because, again, I kind of can't get over how easy this turns out to be. And if you follow the link into Jason's `wireguard-go` code, until you hit gVisor itself, it's not much more complicated under the hood. Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique. The whole th…

> Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique.

Yes! Userspace TCP/IP is how we implement firewall for Androids (which don't expose iptables on non-root devices but let you setup TUN interfaces via VPN APIs). Right now, we rely on LwIP (wrapped in golang) and it has worked wonderfully well; especially since it is light-weight without any locking-overheads (single-threaded) and that bodes well for battery-powered devices.

> The whole thing is kind of a vindication for Go's standard library network interface, which I have always hated.

The Fuchsia team at Google is re-implementing netstack3 in Rust (and hence you're probably right to call it "gVisor netstack") due to what I presume are performance and efficiency reasons (which is of interest to us because we develop for smartphones). Of course, flyctl doesn't need that, but since you wrote about pulling in heavy dependencies, I am interested in your take on it.

Re: SSH and User-Mode IP WireGuard

#43

Post by sdevonoes »

Sounds very cool and all but at the same time it sounds like a terrible thing to maintain in the future.

Perhaps it's just me, but this is something I would accept as a "hey, I was bored and worked on something on my free time. It's probably broken but nobody cares because it's a toy thing, but it's sooo cool". I wouldn't accept it as " Fly.io OKR 1.3 (2021): SSH and User-mode iP WireGuard"... it's sounds pretty much like a hack.

Re: SSH and User-Mode IP WireGuard

#45

Post by tptacek »

sdevonoes wrote: post #43

Sounds very cool and all but at the same time it sounds like a terrible thing to maintain in the future. Perhaps it's just me, but this is something I would accept as a "hey, I was bored and worked on something on my free time. It's probably broken but nobody cares because it's a toy thing, but it's sooo cool". I wouldn't accept it as " Fly.io OKR 1.3 (2021): SSH and User-mode iP WireGuard"... it's sounds pretty much…

This is called "coming to grips with the insanity that is gVisor, the Docker runtime for GKE that is also inexplicably just a Go import". I feel your pain.

Wait until I find a reason to put a whole virtual memory manager into `flyctl`. I'll probably knock out a whole bunch of MBOs that way, and gVisor has me covered.

Re: SSH and User-Mode IP WireGuard

#47

Post by ignoramous »

vlmutolo wrote: post #29

> Normally, this big balloon thingy would be an elaborate scheme to get you to check out our product, but here it's just pointing out some new source code we haven't talked about elsewhere. I really enjoy this style of writing from a company. Regarding the article, it seems like Fly has pulled off some insane networking nonsense, but I don’t know enough about networking yet to understand it. Saving this page for late…

> Regarding the article, it seems like Fly has pulled off some insane networking nonsense

Fly is essentially building a Tailscale-esque infrastructure to service one part of their cloud offering. It is indeed insane the amount of heavy-lifting they do to make it all work. They seem like a cross between packetfabric, gitops, docker, and hashicorp but with way less engineers on the team.

Re: SSH and User-Mode IP WireGuard

#48

Post by mrkurt »

ignoramous wrote: post #47
vlmutolo wrote: post #29

> Normally, this big balloon thingy would be an elaborate scheme to get you to check out our product, but here it's just pointing out some new source code we haven't talked about elsewhere. I really enjoy this style of writing from a company. Regarding the article, it seems like Fly has pulled off some insane networking nonsense, but I don’t know enough about networking yet to understand it. Saving this page for late…

> Regarding the article, it seems like Fly has pulled off some insane networking nonsense Fly is essentially building a Tailscale-esque infrastructure to service one part of their cloud offering. It is indeed insane the amount of heavy-lifting they do to make it all work. They seem like a cross between packetfabric, gitops, docker, and hashicorp but with way less engineers on the team.

To be fair, what Tailscale is doing is much harder than our private networking. They have to deal with NAT, mobile OSes, etc.

We mostly just try to pick the right primitives. And frequently get that wrong. Like that time we wrote our own JS runtime ...

Re: SSH and User-Mode IP WireGuard

#50

Post by ignoramous »

jrockway wrote: post #19
tptacek wrote: post #6

Earlier quoted context omitted.

No, it is extra-super complex infrastructure for a fairly simple thing. Normal SSH still works, and is usually going to be what people end up using. You just have to have WireGuard installed and running. The product feature here is less interesting than how we did it.

It's not clear to me how much day-to-day use of Wireguard being a Fly customer requires, but I can't help but wonder if you guys should collaborate with Tailscale to make all of the micro-VMs appear on a Tailscale network, and authorize access between humans and the VMs that way. (I admit that I haven't looked much into mesh networking / edge servers, so I don't know what the problems are. I always preferred Internet…

> ...I can't help but wonder if you guys should collaborate with Tailscale...

I imagine a merger! Tailscale's mission is to "simplify the long tail of software development", and coincidentally, fly does just that (if only for server-side apps right now).

Post reply on HN