31–40 of 180 posts
> More to come! We’re itching to see just how many different ways this bet might pay off. Or: we’ll perish in flames! Either way, it’ll be fun to watch.
is like nails on the chalkboard for me.
Earlier quoted context omitted.
I used Fly for some projects, I really like it. But once again, for many of my projects, I still need my outbound IPs to resolve to a specific country. I can't have them all resolve to Chicago, US in undeterministic ways. I would be willing to pay an additional cost for this but even with reserved IPs, I am given IPs that are labelled as Chicago, US IPs by GeoIP providers even for non US regions.
fwiw - our network folks _should_ have fixed this a few weeks ago. Some of the outbound IPs were incorrectly tagged in some of the geoip databases as being in the US when they were not.
I'll try again if the situation is different now.
I'm confused about what this is actually offering (also very tired due to some flight problems; anyway) To me, I'd imagine kubernetes on fly as running kind (kubernetes in docker) with fly converting the docker images to firecracker images OR "normal" kubernetes api server running on one machine then using CAPI/or a homegrown thing for spinning up additional nodes as needed. So, what's the deal here? Why k3s + a virt…
You can certainly boot up your own K8s cluster, any way you'd like to, just by enlisting a bunch of Fly Machines and configuring them yourself. A Fly Machine is just a VM, and you have root in the VM. You can set up systemd, you can set up Docker, you can run kubelets on all your Machines. The thought here is: Fly.io already does a lot of the things any K8s distribution would do. If you were to boot up a complete K8s…
> To keep things simple, we used Nomad, and instead of K8s CNIs, we built our own Rust-based TLS-terminating Anycast proxy (and designed a WireGuard/IPv6-based private network system based on eBPF). That is quite the opposite of “simple”. That is in fact, overly complex and engineered.
What part of it is overly complex and engineered? Maybe you're right, but it's hard to respond without a better idea of what you think our problem domain was.
This is impressive, but also seems to fly in the face of their raison d'etre. I don't even bother with k8s on AWS because it's too complex for even a mid-size operation. Isn't the point of PaaS to obscure complexity?
Earlier quoted context omitted.
You can certainly boot up your own K8s cluster, any way you'd like to, just by enlisting a bunch of Fly Machines and configuring them yourself. A Fly Machine is just a VM, and you have root in the VM. You can set up systemd, you can set up Docker, you can run kubelets on all your Machines. The thought here is: Fly.io already does a lot of the things any K8s distribution would do. If you were to boot up a complete K8s…
Did you consider using/adopting the seemingly defunct nomad virtual kubelet?
> To keep things simple, we used Nomad, and instead of K8s CNIs, we built our own Rust-based TLS-terminating Anycast proxy (and designed a WireGuard/IPv6-based private network system based on eBPF). That is quite the opposite of “simple”. That is in fact, overly complex and engineered.
What part of it is overly complex and engineered? Maybe you're right, but it's hard to respond without a better idea of what you think our problem domain was.
Earlier quoted context omitted.
Did you consider using/adopting the seemingly defunct nomad virtual kubelet?
Do people want to run Nomad on Fly.io? It wouldn't have addressed any of the reasons we replaced Nomad with flyd; it's hard to scale a globally synchronized distributed database for real-time scheduling.
Earlier quoted context omitted.
What part of it is overly complex and engineered? Maybe you're right, but it's hard to respond without a better idea of what you think our problem domain was.
Reading the features of your CNI, I don't see why Calico wouldn't have worked for your needs, what made you want to DIY?
I'm confused about what this is actually offering (also very tired due to some flight problems; anyway) To me, I'd imagine kubernetes on fly as running kind (kubernetes in docker) with fly converting the docker images to firecracker images OR "normal" kubernetes api server running on one machine then using CAPI/or a homegrown thing for spinning up additional nodes as needed. So, what's the deal here? Why k3s + a virt…
You can certainly boot up your own K8s cluster, any way you'd like to, just by enlisting a bunch of Fly Machines and configuring them yourself. A Fly Machine is just a VM, and you have root in the VM. You can set up systemd, you can set up Docker, you can run kubelets on all your Machines. The thought here is: Fly.io already does a lot of the things any K8s distribution would do. If you were to boot up a complete K8s…
Not sure what the implications of that are in practice but sounds interesting.