51–60 of 180 posts
Wouldn't it have cost less to enhance the Nomad scheduler rather than move to, and enhance, Kubernetes? This aside, Fly is in a position to build its own alternative to K8s and Nomad from scratch, so maybe it will?
Wouldn't it have cost less to enhance the Nomad scheduler rather than move to, and enhance, Kubernetes? This aside, Fly is in a position to build its own alternative to K8s and Nomad from scratch, so maybe it will?
The ordinary way someone would boot up an app on Fly.io is to visit a directory in their filesystem with a Rails or Django or Express app or something, or a Dockerfile, and just type `flyctl launch`. No K8s will be involved in any way. You have to go out of your way to get K8s on Fly.io. :)
Am I understanding correctly that because they map a “Pod” to a “Fly Machine”, there’s no intermediate “Node” concept? If so, this is very attractive. When using GKS, we had to do a lot of work to get our Node utilization (the percent of resources we had reserve on a VM actually occupied by pods) to be higher than 50%. Curios what happens when you run “kubectl get nodes” - does it lie to you, or call each region one…
I wonder how it copes with things like anti-affinity rules, where you don't want two things running on the same physical / virtual server for resilience reasons.
Earlier quoted context omitted.
Why? If you're using something like Fly you should 100% always have a fallback plan ready. You are gambling using smaller players to get cheaper services or some other benefit the big players don't offer in exchange for the very real possibility of a random day they announce 30 days til they permanently shutdown with zero migration path. I don't think it's in poor taste to acknowledge exactly what everyone should und…
You can communicate those ideas specifically without hiding it beneath the veneer of relatability. The entire post started with this bit: > But, come on: you never took us too seriously about K8s, right? K8s is hard for us to use, but that doesn’t mean it’s not a great fit for what you’re building. We’ve been clear about that all along, right? Sure we have! which already starts the post in a bad space for the reader.…
Earlier quoted context omitted.
I wonder how it copes with things like anti-affinity rules, where you don't want two things running on the same physical / virtual server for resilience reasons.
You wouldn’t use affinity rules anymore. The pods are scheduled on a single virtual-kubelet node, so if you use anti-affinity scheduling would fail.
> A Fly Volume is a slice of an NVMe drive on the physical server your Fly App runs on. It’s tied to that hardware.
Does the k8s have any kind of storage provisioning that allows pods with persistent storage (e.g. databases) to just do their thing without me worrying about it or do I still need to handle disks potentially vanishing?
I think this is the only hold-up that stops me actually using Fly. I don't know what happens if my machine crashes and is brought back on different hardware. Presumably the data is just not there anymore.
Is everyone else using an off-site DB like Planetscale? Or just hoping it's an issue that never comes up, w/ backups just in case? Or maybe setting up full-scale DB clusters on Fly so it's less of a potential issue? Or 'other'?
Is this still a limitation for Fly k8s? > A Fly Volume is a slice of an NVMe drive on the physical server your Fly App runs on. It’s tied to that hardware. Does the k8s have any kind of storage provisioning that allows pods with persistent storage (e.g. databases) to just do their thing without me worrying about it or do I still need to handle disks potentially vanishing? I think this is the only hold-up that stops m…
We back up volumes to off-net block storage, and, under the hood, we can seamlessly migrate a volume to another physical (the way we do it is interesting, and we should write it up, but it's still also an important part of our work sample hiring process, which is why we haven't). So your app could move from one physical to another; the data would come with it.
On the other hand: Fly Volumes are attached storage. They're not a SAN system like EBS, they're not backed onto a 50-9s storage engine like S3. If a physical server throws a rod, you can lose data. This is why, for instance, if you boot up a Fly Postgres cluster here and ask us to do it with only one instance, we'll print a big red warning. (When you run a multi-node Postgres cluster, or use LiteFS Cloud with SQLite, you'd doing at the application layer what a more reliable storage layer would do at the block layer).
Earlier quoted context omitted.
You wouldn’t use affinity rules anymore. The pods are scheduled on a single virtual-kubelet node, so if you use anti-affinity scheduling would fail.
If there were a virtual kubelet per unit of granularity (datacenter, in their case?) then you would be able to use affinity rules just fine.
The press release states that your k8s API is actually running on a single machine with k3s and a virtual-kubelet. So, I’m not sure if it’s one “cluster” per region, or one “cluster” with multiple virtual-kubelets for regions.
Either way, your FKS cluster control-plane would sit in a single region.
Am I understanding correctly that because they map a “Pod” to a “Fly Machine”, there’s no intermediate “Node” concept? If so, this is very attractive. When using GKS, we had to do a lot of work to get our Node utilization (the percent of resources we had reserve on a VM actually occupied by pods) to be higher than 50%. Curios what happens when you run “kubectl get nodes” - does it lie to you, or call each region one…
The node would be a virtual-kubelet. You can check out the virtual-kubelet GitHub repo for more info. Interestingly, there are already multiple providers of virtual-kubelet. For example, Azure AKS has virtual nodes where pods are Azure Container Instances. There’s even a Nomad provider. > So that’s what we do. When you create a cluster, we run K3s and the Virtual Kubelet on a single Fly Machine. So probably a cluster…