111–120 of 169 posts
I always love the style how fly.io's blog is written, and a big fan of their freemium product. I am see a group of enthusiastic hackers behind the product, and keeps improving it in a reasonable way, or a cool way that doesn't sound boring.
I like it too but I feel like they overdo it just a tad.
1. Could you include more info on which userspace tcp/ip stack you use and why? I presume doing userspace UDP is relatively trivial/fast compared to what slirp had to do with tcp.
2. How does flyctl hijack syscalls across Linux and Windows? Is there some abstraction to do that? Wasn't even aware this was a pattern on Windows.
I realize I could read the code, but would appreciate some direction.
Kudos for the websocket proxy too! Would be really cool if this and the unprivileged wireguard became standard parts of wireguard toolset.
First off, you guys rule for doing this. I been dreaming about a general purpose userspace/unprivileged wireguard wrapper, this gets us closer to that. 1. Could you include more info on which userspace tcp/ip stack you use and why? I presume doing userspace UDP is relatively trivial/fast compared to what slirp had to do with tcp. 2. How does flyctl hijack syscalls across Linux and Windows? Is there some abstraction t…
I could try to give you reasons we use it but the truth is that I mentioned something about wanting a user-mode TCP and Jason Donenfeld said netstack was there, and then got it to work.
We don't do any system call hijacking! Don't have to. Go abstracts Dialers and Listeners, and the WireGuard part itself is just a vanilla UDP protocol spoken over a PacketConn.
Fly.io's blog posts are incredible, they really seem to really enjoy what they do and want to share what they've made with everyone else. I love them for that. I wish that more companies could be like this and skip the corporate BS, it shows that they really have something outstanding to offer.
A lot of (small–medium sized, tech) companies just don’t have a process to get things out on their blog like this. It might be that only a few senior people have the ability to write posts and they are not interested or busy with other things, or it might be that there is a slow review process for posts that makes writing them unpleasant, or it might be that they don’t want to reveal IP or have an opinion and so have…
How do they manage their mesh? I've just been doing research on setting up my own wireguard mesh (currently using a spoke/hub setup with pi-hole/pivpn). I found https://github.com/HarvsG/WireGuardMeshes today which is awesome, but I'm curious what fly.io / other readers here may be using.
I usually build my own solutions, but I've played with Netmaker and it seems solid. https://github.com/gravitl/netmaker
Earlier quoted context omitted.
The problem isn't new but the previous best practices involved giving the tool super user. Usually through an installation process. See most other VPNs. UserMode TCP/IP stacks aren't very common in practice. This is why what fly.io did is interesting.
Or network namespaces? User-space tcp might also make it easier to do tcp checkpoint restore and container/app migration. Interesting write-up. I keep thinking of the nightmare of keeping up with the world of Internet middleboxens, broken net layer implementations and icmp hacks that the Linux kernel supports and makes 'just work'. The jump to usermode tcp seems interesting if you're not worried about that (and I've…
Earlier quoted context omitted.
Replacing SSL/TLS with wireguard is cool but aren't you just going to run into the same headaches of rotating certificates/keys? No one is really going to rely on using the same wireguard key indefinitely, right?
It's pretty easy for us to rotate keys now, since new WireGuard peers are extremely cheap to bring up (part of the point of the post is that for most of the last year, that was the opposite of the case, and a new peer was a very painful thing to ask for). But rotating WireGuard keys with Fly.io makes about as much sense as rotating the OAuth2 API token `flyctl` uses (the token is strictly more powerful than the WireG…
Earlier quoted context omitted.
This is where the article lost me a little bit. I (think I) technically got the part of running a TCP/IP stack in an unprivileged user process, so you don't have to elevate privilege for adding a network interface and using the host OS TCP/IP stack. And maybe that's already very cool. But: - What other benefits does it give you? - This isn't a new problem and presumably has prior best practices for mitigation. What i…
This is a good question and part of the reason you didn't get a clear answer from the article is that I'm not sure if I have a clear answer. What I think user-mode TCP/IP gives us is the ability to build arbitrary services --- Postgres, Redis, SSH, network management, whatever --- without having to make infrastructure changes. We don't have to have some weird API or application proxy that knows what's running and who…
Sure, just how many distinct WireGuard configurations would the gateways be comfortable with per-Fly app? 1K+? 100K+? 1M+?
So we need assured guarantees that data is not traversing outside of the region. most cloud vendors have specific data residency compliance for India - https://aws.amazon.com/compliance/india-data-protection/
Earlier quoted context omitted.
So do you think I'm being too extreme on this? Or did you just implement SSH because customers want it?
I think it's sensible to run an application fleet without SSH access, but it's tough for a hosting provider that has to support lots of different application fleets to not offer a way to get a shell. Our authorization systems are about to get sharply more interesting as we roll out Macaroon-style tokens this quarter, so I'm optimistic we'll get to a place where we make both styles of application owners happy. I was m…