Live data from Hacker News

SSH and User-Mode IP WireGuard

fly.io

61–70 of 104 posts

Re: SSH and User-Mode IP WireGuard

#61
This is fantastic. I maintain a list[0] of tunneling software. One of the few downsides of WireGuard is the inability to run it in unprivileged situations. The complexity and performance overhead here might still be too much to edge out solutions like SSH tunnels, but I love that the space is being explored.

I'm hopeful we'll also see some robust QUIC-based tunneling tools over the next couple years.

[0]: https://github.com/anderspitman/awesome-tunneling

Re: SSH and User-Mode IP WireGuard

#62
post #2

I added some example code to the post, because, again, I kind of can't get over how easy this turns out to be. And if you follow the link into Jason's `wireguard-go` code, until you hit gVisor itself, it's not much more complicated under the hood. Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique. The whole th…

> Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique. Yes! Userspace TCP/IP is how we implement firewall for Androids (which don't expose iptables on non-root devices but let you setup TUN interfaces via VPN APIs). Right now, we rely on LwIP (wrapped in golang) and it has worked wonderfully well; especially sin…

Don't want to go OT but I'm super curious what your experience developing a network application for non-root Android devices has been?

As a non-Android developer, I've been working on a project the last few months that involves running an HTTP server on the device and tunneling out so it can receive requests from the outside world, and the platform feels nerfed at every level from filesystem access to keeping your server from being battery-killed.

Re: SSH and User-Mode IP WireGuard

#63
post #60
post #59

Earlier quoted context omitted.

Why is userland TCP/IP stack needed? I didn't get this part of the story.

Work through it. You need WireGuard to talk to SSH on our instances; that can't change, it's a security rule. You can get userland WireGuard; that's how most people WireGuard. But you can't create an OS tun device: you need root to do that; you might as well just install WireGuard. Ok: you handshaked a WireGuard connection in Go. What's next? Let's simplify it: from your Go WireGuard connection, just do an HTTP GET.…

I think I got it now!

I was confused because Tailscale does not bring its own userland TCP/IP. It can - as a VPN solution - rely on OS-provided TCP/IP stack, but you wanted to avoid having to hook up flyctl into OS as a virtual network interface, right?

Re: SSH and User-Mode IP WireGuard

#65
post #63
post #60

Earlier quoted context omitted.

Work through it. You need WireGuard to talk to SSH on our instances; that can't change, it's a security rule. You can get userland WireGuard; that's how most people WireGuard. But you can't create an OS tun device: you need root to do that; you might as well just install WireGuard. Ok: you handshaked a WireGuard connection in Go. What's next? Let's simplify it: from your Go WireGuard connection, just do an HTTP GET.…

I think I got it now! I was confused because Tailscale does not bring its own userland TCP/IP. It can - as a VPN solution - rely on OS-provided TCP/IP stack, but you wanted to avoid having to hook up flyctl into OS as a virtual network interface, right?

I think you've got it. Tailscale is installing WireGuard. You have to have privileges to install Tailscale. They can tell the OS to route packets through their virtual interface.

We could too! This is all in `wireguard-go`. But we'd have to prompt users to escalate privileges every time they tried to SSH somewhere (or, worse, install a long-term resident thingy, just to SSH to things). We don't want to own your VPN connections!

This is an end-run around all of that; we just take responsibility for all of TCP/IP, in our dumb little command line program.

Re: SSH and User-Mode IP WireGuard

#66
> I’ve written a bunch about private networking at Fly. Long story short: it’s like a simpler, IPv6 version of GCP or AWS “Virtual Private Clouds”; we call it “6PN”. When an app instance (a Firecracker micro-VM) is started at Fly, we assign it a special IPv6 prefix; the prefix encodes the app’s ID, the ID of its organization, and an identifier for the Fly hardware it’s running on. We use a tiny bit of eBPF code to statically route those IPv6 packets along our internal WireGuard mesh, and to make sure that customers can’t hop into different organizations.

My first thought was "Wow, can we make this _more_ complicated please?", and then I read the rest of the post.

I hate technology.

Re: SSH and User-Mode IP WireGuard

#67
post #2

I added some example code to the post, because, again, I kind of can't get over how easy this turns out to be. And if you follow the link into Jason's `wireguard-go` code, until you hit gVisor itself, it's not much more complicated under the hood. Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique. The whole th…

> The whole thing is kind of a vindication for Go's standard library network interface, which I have always hated.

Curious about this. I've generally found Go's net libs to be pretty pleasant. Can you compare/contrast it with others you like better?

Re: SSH and User-Mode IP WireGuard

#68

Amazing. The client API is profoundly simple. Also, this post prompted me to look closer at Fly.io, and it's leapfrogged to the top of my shortlist for an imminent client "edge proxy" project.

I love proxies and think all problems should be solved with proxies. Which means – if you give Fly.io a try and need any help, you should let me know!

Re: SSH and User-Mode IP WireGuard

#69
post #51

This is such an interesting marketing strategy, I had never thought of selling B2B production infrastructure under the aesthetic of, “Can you believe this shit actually works?”

Yes! This person gets it.

I'm a fan of the writing style. It reminds of smart people I know. I haven't bought any fly.io yet, so I don't know that I'm you're target market. Still--well said, repeatedly.

Re: SSH and User-Mode IP WireGuard

#70

Earlier quoted context omitted.

> Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique. Yes! Userspace TCP/IP is how we implement firewall for Androids (which don't expose iptables on non-root devices but let you setup TUN interfaces via VPN APIs). Right now, we rely on LwIP (wrapped in golang) and it has worked wonderfully well; especially sin…

Don't want to go OT but I'm super curious what your experience developing a network application for non-root Android devices has been? As a non-Android developer, I've been working on a project the last few months that involves running an HTTP server on the device and tunneling out so it can receive requests from the outside world, and the platform feels nerfed at every level from filesystem access to keeping your se…

> ...and the platform feels nerfed at every level

Android development is a bit tedious relatively compared to iOS due to having to support multiple API levels and having to account for subtleties across OEM implementations, but things have drastically improved in the last few years, especially after Oreo (Android 8).

> ...from filesystem access

Watch out for tutorials still recommending workarounds that aren't necessarily needed due to Jetpack and friends: https://developer.android.com/modern-android-development.

> ...to keeping your server from being battery-killed.

See: https://dontkillmyapp.com/

Process reaping is also, I believe, a problem on iOS? One way to keep a process out of OutOfMemory/LowMemoryKiller's reach is to make it a foreground service (what stuff like Music Players do) and generally be very stringent with resource use. It is easy to profile for resource usage thanks to Android Studio's built-in profiler and tools like https://perfetto.dev/

Post reply on HN