This looks interesting, but I run a bare metal k8s cluster over wire guard for independence. Not willing to rely on a nonstandard api/platform. Current provider annoys me and I’m shutting down nodes the next day. Probably could not do that on FKS.
Fly Kubernetes
121–130 of 180 posts
Re: Fly Kubernetes
#122Why should one use kubernetes? Or rather, at what point of an apps growth cycle does k8s become appropriate?
I'd say, not in an app's growth cycle, but when an organization wants to manage and scale platforms for itself, on which it runs apps, is when k8s becomes appropriate. In other words, k8s is a platform builder.
If the org is going to go that far with managing and scaling their platform, why would they use fly then?
Re: Fly Kubernetes
#123Earlier quoted context omitted.
Is that what we did here? You get that this is just a `flyctl` feature and some Dockerfiles, right? You could have built FKS yourself by forking `flyctl`.
Maybe you were not wrong on technical merits, but your move proves you underestimated the value of the Kubernetes ecosystem of lots and lots of standards and trained talent. It’s like building a fancy CMS today with a fancy API, but then make it Wordpress compatible when you realize the value of WP is not just in the tech, but also in the ecosystem.
[deleted]
Re: Fly Kubernetes
#124Earlier quoted context omitted.
Are any of the cloud provided managed K8s offerings just K8s under the covers? I’ve always assumed all of them shim the K8s api onto other more bespoke orchestration systems.
Most are pretty much actually k8s, though they tend to have different ways of handling the masters, my understanding is that it's the same k8s binaries and code What I've seen is the k8s APIs working their way into other systems like cloud functions and servers, so you can use the same Yaml across vendors and products. This is further solidifying k8s APIs as an industry standard Searching "managed kubernetes provider…
I’ve looked extensively at the documentation for gke autopilot (a system I use extensively) and haven’t found any documentation on how they orchestrate those clusters under the covers.
I’ve always assumed it was a plugin to borg, not a different fleet orchestrator. Not to be overtly needy, but do you have a link to docs that contradicts that?
Re: Fly Kubernetes
#125How does this handle multiple containers for a Pod? In a container runtime k8s, containers within a pod share the same network namespace (same localhost) and possibly pid namespace. The press release maps pods to machines, but provides no mapping of pod containers to a Fly.io concept. Are multiple containers allowed? Do they share the same network namespace? Is sharing PID namespace optional? Having multiple containe…
Why should you do this - sounds like an antipattern to me
Re: Fly Kubernetes
#126How does this handle multiple containers for a Pod? In a container runtime k8s, containers within a pod share the same network namespace (same localhost) and possibly pid namespace. The press release maps pods to machines, but provides no mapping of pod containers to a Fly.io concept. Are multiple containers allowed? Do they share the same network namespace? Is sharing PID namespace optional? Having multiple containe…
You can use mount namespaces, or even containers in your VM. Maybe that's how?
Fly.io claims it’s “just a VM”. But, Fly.io Machines are an abstraction of microVMs using Firecracker. Building upon that, the FKS implementation is an abstraction on top of Fly.io Machines. So what I’m asking is how, if even, does the FKS implementation support multiple containers for a pod? Using FKS, the abstraction is no longer a VM.
It seems that Fly.io Machines support multiple processes for a single container, but not multiple containers per Machine [0]. This means one container image per Machine and thus no shared network namespace across multiple containers.
Re: Fly Kubernetes
#127How does this handle multiple containers for a Pod? In a container runtime k8s, containers within a pod share the same network namespace (same localhost) and possibly pid namespace. The press release maps pods to machines, but provides no mapping of pod containers to a Fly.io concept. Are multiple containers allowed? Do they share the same network namespace? Is sharing PID namespace optional? Having multiple containe…
Why should you do this - sounds like an antipattern to me
It’s used widely in the Kubernetes world and is known as sidecars [0].
[0] https://kubernetes.io/blog/2023/08/25/native-sidecar-contain...
Re: Fly Kubernetes
#128Earlier quoted context omitted.
You can use mount namespaces, or even containers in your VM. Maybe that's how?
Fly.io claims it’s “just a VM”. But, Fly.io Machines are an abstraction of microVMs using Firecracker. Building upon that, the FKS implementation is an abstraction on top of Fly.io Machines. So what I’m asking is how, if even, does the FKS implementation support multiple containers for a pod? Using FKS, the abstraction is no longer a VM. It seems that Fly.io Machines support multiple processes for a single container,…
You can run Docker on a Fly Machine, and run arbitrary numbers of containers inside of it. Or you can run lots of small Fly Machines. FKS is just one model for deploying things.
Re: Fly Kubernetes
#129I like the discussion on scheduling. One of the things I've thought recently is that, since there's no one model of how an app or system should work, nor one network architecture, there shouldn't be one scheduler. Instead, I think the system components should expose themselves as independent entities, and grant other system components the ability to use them under criteria. With this model, any software which can use…
> I know corporations hate standards and love to make their own little one-offs, but it's really holding back technological progress.
Corporations create standards all the time, either directly or through standards bodies, that they also fund. You can already push logs with syslog, or transform them with Beats then push them; you can already attach storage from elsewhere, etc etc. It's just often a bad idea to for performance and data movement cost reasons.
I don't see the major technological progress this holds back, and if you think technological progress is a measure of how much corporations hate standards, then by that logic, based on the last 50 years of utterly insane progress, they must love standards.
Re: Fly Kubernetes
#130Earlier quoted context omitted.
I wonder how it copes with things like anti-affinity rules, where you don't want two things running on the same physical / virtual server for resilience reasons.
You wouldn’t use affinity rules anymore. The pods are scheduled on a single virtual-kubelet node, so if you use anti-affinity scheduling would fail.
> You wouldn’t use affinity rules anymore
Point being: what if I wanted to do this? How could I achieve making sure services were running according to the antiaffinity rules I provided? E.g. not on same physical machine; not on same VM; not in same datacentre; not in same region; etc.