Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our User-Mode WireGuard Year

fly.io

Post reply on HN
Print view View original

101–110 of 169 posts

Re: Our User-Mode WireGuard Year

#101

Post by atonse »

michaeldwan wrote: post #91
ericpauley wrote: post #55

Earlier quoted context omitted.

We recently moved our entire app deployment over to Fly and are mostly loving it, but one of the mildly janky features is hallpass. For instance, (1) connections often fail if you have X forwarding enabled (even if you did no specifial config on the machine), and (2) port forwarding doesn't work. While these aren't really a big deal since (1) you can just disable X forwarding in ssh_config and (2) port forwarding is…

As for autoscaling, our hands are tied as long as we're running on Nomad. Right now our autoscaler is nothing more than some ruby that loops over data from prometheus and changes counts in Nomad. It's slow and buggy, but worse we don't have control over where Nomad places VMs or which ones it stops when scaling down. We're working on a replacement for Nomad (called flyd) that gives us full control over VMs. Once apps…

I’d love to hear more about this move away from Nomad.

We haven’t had too good a time with nomad, but not sure if it’s just our limited understanding. It doesn’t help that there are very few people out there that know it.

Re: Our User-Mode WireGuard Year

#102

Post by dudus »

mayli wrote: post #93

I always love the style how fly.io's blog is written, and a big fan of their freemium product. I am see a group of enthusiastic hackers behind the product, and keeps improving it in a reasonable way, or a cool way that doesn't sound boring.

I like it too but I feel like they overdo it just a tad.

Re: Our User-Mode WireGuard Year

#103

Post by pier25 »

Not even 30 mins ago I set up WireGuard to connect to a PG instance on Fly.

I expected this to be a headache but it took less than 5 mins to download WG, generate the conf with the fly CLI and paste it into WG. Done.

Re: Our User-Mode WireGuard Year

#104

Post by amscanne »

bradfitz wrote: post #60
amscanne wrote: post #40

Earlier quoted context omitted.

I think Tailscale uses user-mode TCP/IP (also gVisor netstack) for some client devices, like iOS? But could be wrong here.

We use it on all platforms _except_ iOS, for binary size/memory reasons. (iOS 15 bumped the Network Extension memory limit to 50 MB, but we still need to be super trim for iOS 14's 15 MB limit)

LOL, I was precisely wrong.

Is there actually a preference for user-mode networking? I assume that’s primarily about control and flexibility?

Either way, I hope that the PacketBuffer changes can help reduce footprint after issues are shaken out.

Re: Our User-Mode WireGuard Year

#105

Post by est »

Been using tunsafe for years. It's solid wireguard-compatible, works on user-space, and had extra features like TCP handshake+UDP data, https obfuscation, etc.

Author is @strigeus of uTorrent/Spotify fame.

Re: Our User-Mode WireGuard Year

#106

Post by xarope »

mrkurt wrote: post #73
mbesto wrote: post #67

Earlier quoted context omitted.

> I wish that more companies could be like this and skip the corporate BS, it shows that they really have something outstanding to offer. The nature of the blog typically cater towards the intended audience. The CIO of Disney doesn't give a sh*t if the protocol is called WireGuard or OpenVPN or that if it uses AES-256 encryption - he/she wants someone to tell them that their developers are securely accessing their in…

For better or worse, I can guarantee you that we won't ever write articles for the Disney CIO. Unless I get fired. Whitepapers. They want whitepapers and magic quadrants.

White papers are to CxOs what TED talks are for the uninformed, an easy way to get someone up to speed on a highly complicated subject, and more fool them (meaning the uninformed) if they think it means they are now an expert, which sadly a lot do (think they are now an expert).

I kind of miss the IBM ITSO Redbooks. No idea if they still maintain the same quality today, but in the 90s and pre-internet/google/wiki etc, they were fonts of deep knowledge.

Re: Our User-Mode WireGuard Year

#107

Post by hedora »

bityard wrote: post #71
ulzeraj wrote: post #64

I was using wireguard-go on FreeBSD jail running on top of an APU2C2 board. Torrenting from my laptop caused wireguard-go cpu usage to spike to high loads and 30-50% CPU usage. Loading wireguard-kmod on the host machine plus some devfs rules dropped the CPU load to 0s. Not sure what happened there. The processor seems to score less than an RPi4 on Geekbench.

I use one of these as a firewall (running OPNSense) and they're very nice but the CPU is indeed _slow_. It's plenty good enough for everything the firewall does but booting it up takes minutes and that's saying something for FreeBSD.

Odd. I run openbsd on a similar one, and booting is reasonably fast. I even have a linux vm running on it in vmd, and haven’t noticed performance issues with that either.

Re: Our User-Mode WireGuard Year

#108

Post by michaeldwan »

atonse wrote: post #101
michaeldwan wrote: post #91

Earlier quoted context omitted.

As for autoscaling, our hands are tied as long as we're running on Nomad. Right now our autoscaler is nothing more than some ruby that loops over data from prometheus and changes counts in Nomad. It's slow and buggy, but worse we don't have control over where Nomad places VMs or which ones it stops when scaling down. We're working on a replacement for Nomad (called flyd) that gives us full control over VMs. Once apps…

I’d love to hear more about this move away from Nomad. We haven’t had too good a time with nomad, but not sure if it’s just our limited understanding. It doesn’t help that there are very few people out there that know it.

We'll write about it when the time comes. To be fair, Nomad and Consul have served us well. Most of our troubles stem from abusing them in ways they weren't designed to handle.

Re: Our User-Mode WireGuard Year

#109

Post by tptacek »

mobilio wrote: post #99
anderspitman wrote: post #14

Usermode WireGuard would be a big deal. I maintain a list[0] of tunneling solutions, and one of the only limitations of systems built on WireGuard is the requirement for admin privileges. Even with the performance hit from running outside the kernel, UDP-based tunnels have a lot of advantages for multiplexing channels. Pretty much your only mainstream options today are QUIC and WireGuard, and only QUIC is intended to…

TunSafe also runs on userspace: https://github.com/TunSafe/TunSafe

It looks like from the source code that TunSafe opens up a tunnel device, in which case it's doing TCP/IP in the kernel, not in userland.

Re: Our User-Mode WireGuard Year

#110

Post by tptacek »

pier25 wrote: post #103

Not even 30 mins ago I set up WireGuard to connect to a PG instance on Fly. I expected this to be a headache but it took less than 5 mins to download WG, generate the conf with the fly CLI and paste it into WG. Done.

You shouldn't have to do this unless you want to (for instance, to make a permanent WireGuard connection) --- you can just run `flyctl proxy` to set up a connection to 5432/tcp.
Post reply on HN