Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fly Kubernetes

fly.io

Post reply on HN
Print view View original

131–140 of 180 posts

Re: Fly Kubernetes

#131

Post by postalrat »

javaunsafe2019 wrote: post #125
kuhsaft wrote: post #110

How does this handle multiple containers for a Pod? In a container runtime k8s, containers within a pod share the same network namespace (same localhost) and possibly pid namespace. The press release maps pods to machines, but provides no mapping of pod containers to a Fly.io concept. Are multiple containers allowed? Do they share the same network namespace? Is sharing PID namespace optional? Having multiple containe…

Why should you do this - sounds like an antipattern to me

Its because you have multiple processes (containers) that work together in their little pod. You could stick them all in a single image somehow but that would be much more work and less flexible.

Re: Fly Kubernetes

#132

Post by kuhsaft »

tptacek wrote: post #128
kuhsaft wrote: post #126

Earlier quoted context omitted.

Fly.io claims it’s “just a VM”. But, Fly.io Machines are an abstraction of microVMs using Firecracker. Building upon that, the FKS implementation is an abstraction on top of Fly.io Machines. So what I’m asking is how, if even, does the FKS implementation support multiple containers for a pod? Using FKS, the abstraction is no longer a VM. It seems that Fly.io Machines support multiple processes for a single container,…

You can run Docker on a Fly Machine, and run arbitrary numbers of containers inside of it. Or you can run lots of small Fly Machines. FKS is just one model for deploying things.

Right, but what is the point of FKS then? It’s no longer Kubernetes if it doesn’t support a core behavior of Kubernetes.

If you only support deploying single containers with single processes on FKS, then you might as well use flyctl.

It’s a solvable issue of course. The virtual-kubelet implementation would need to create a Machine running a container runtime image that would then run a the pod containers to match the pod configuration.

I think that some disclaimers of the limitations of FKS compared to standardized Kubernetes should be present and highly visible.

Re: Fly Kubernetes

#133

Post by wavemode »

0xbadcafebee wrote: post #6

I like the discussion on scheduling. One of the things I've thought recently is that, since there's no one model of how an app or system should work, nor one network architecture, there shouldn't be one scheduler. Instead, I think the system components should expose themselves as independent entities, and grant other system components the ability to use them under criteria. With this model, any software which can use…

Obligatory https://xkcd.com/927/

Re: Fly Kubernetes

#134

Post by tptacek »

kuhsaft wrote: post #132
tptacek wrote: post #128

Earlier quoted context omitted.

You can run Docker on a Fly Machine, and run arbitrary numbers of containers inside of it. Or you can run lots of small Fly Machines. FKS is just one model for deploying things.

Right, but what is the point of FKS then? It’s no longer Kubernetes if it doesn’t support a core behavior of Kubernetes. If you only support deploying single containers with single processes on FKS, then you might as well use flyctl. It’s a solvable issue of course. The virtual-kubelet implementation would need to create a Machine running a container runtime image that would then run a the pod containers to match the…

The article discusses what you get by using K8s alongside Fly.io. If you want to bin-pack containers onto Fly Machines, you can of course just boot up your own K8s cluster here; that has always been an option.

Re: Fly Kubernetes

#135

Post by kuhsaft »

tptacek wrote: post #134
kuhsaft wrote: post #132

Earlier quoted context omitted.

Right, but what is the point of FKS then? It’s no longer Kubernetes if it doesn’t support a core behavior of Kubernetes. If you only support deploying single containers with single processes on FKS, then you might as well use flyctl. It’s a solvable issue of course. The virtual-kubelet implementation would need to create a Machine running a container runtime image that would then run a the pod containers to match the…

The article discusses what you get by using K8s alongside Fly.io. If you want to bin-pack containers onto Fly Machines, you can of course just boot up your own K8s cluster here; that has always been an option.

It should discuss what you don’t get compared to the standard behaviors of Kubernetes.

Re: Fly Kubernetes

#136

Post by jipperinbham »

kuhsaft wrote: post #132
tptacek wrote: post #128

Earlier quoted context omitted.

You can run Docker on a Fly Machine, and run arbitrary numbers of containers inside of it. Or you can run lots of small Fly Machines. FKS is just one model for deploying things.

Right, but what is the point of FKS then? It’s no longer Kubernetes if it doesn’t support a core behavior of Kubernetes. If you only support deploying single containers with single processes on FKS, then you might as well use flyctl. It’s a solvable issue of course. The virtual-kubelet implementation would need to create a Machine running a container runtime image that would then run a the pod containers to match the…

We're actively working on the ability to run multiple processes with different images because it's something people using our platform want and it just happens to also be something needed for us to make FKS a more standardized Kubernetes offering.

Re: Fly Kubernetes

#137

Post by dilyevsky »

kasey_junk wrote: post #124
verdverm wrote: post #115

Earlier quoted context omitted.

Most are pretty much actually k8s, though they tend to have different ways of handling the masters, my understanding is that it's the same k8s binaries and code What I've seen is the k8s APIs working their way into other systems like cloud functions and servers, so you can use the same Yaml across vendors and products. This is further solidifying k8s APIs as an industry standard Searching "managed kubernetes provider…

I’ve looked extensively at the documentation for gke autopilot (a system I use extensively) and haven’t found any documentation on how they orchestrate those clusters under the covers. I’ve always assumed it was a plugin to borg, not a different fleet orchestrator. Not to be overtly needy, but do you have a link to docs that contradicts that?

It is definitely not a plugin for borg. Borg is totally different api (source - i was borg sre). Afaik it’s actually vanilla k8s apiserver with some shimmed bespoke storage but it’s not really documented anywhere. You can test that fact using kubectl proxy though

Re: Fly Kubernetes

#138

Post by btown »

tptacek wrote: post #104
benpacker wrote: post #102

Earlier quoted context omitted.

If this is “free GKE autopilot” (autopilot billed at the same price as regular Fly Machine compute), then that changes the way I think about Fly’s basic compute pricing a lot. I would think they should highlight that a lot more in the product announcement!

Say more! What should we highlight more?

As someone not familiar with Fly's offering (but very interested for the same reasons as the post you're replying to!), a couple things come to mind if you're looking at convincing people familiar with k8s to move workloads here:

- https://fly.io/docs/ doesn't show any results when searching kubernetes or k8s or k3s.

- https://fly.io/blog/fks/ is self-admittedly snarky but also doesn't provide details about the product itself. It jumps straight into technical details - and while I like the openness about fault tolerance, there's no paragraph after the intro about what Fly Kubernetes is.

- What exactly does the combination of k3s and virtual-kubelet provide compared to standard k8s? Does it provide secret and confmap storage and namespaces and all those expected things? Can we run things like the Kubernetes dashboard? cert-manager? nginx-ingress?

- On that note, what's the ingress story in general? Is Fly automatically routing traffic to the k8s cluster based on the ingress declarations? Are there limitations? Where are they documented?

- Most people running k8s will have fault-tolerant workloads, but reasonable expectations for pod lifetime and reliability of underlying "hardware" are nonetheless important. If I'm migrating from EKS or GKE and want to run a 24/7 background process, can I expect it to keep running on the same Fly Machine for weeks or months until updated? Or are there limits here? (This might be better documented for Fly Machine but it's worth documenting specifically in this context.)

Absolutely understand that this is an experimental work in progress. It's really cool work! But it's also impossible to even justify playing with as an experiment, with so many unanswered questions about where hard caps in the functionality may be hit.

Re: Fly Kubernetes

#139

Post by 0xbadcafebee »

verdverm wrote: post #26
0xbadcafebee wrote: post #6

I like the discussion on scheduling. One of the things I've thought recently is that, since there's no one model of how an app or system should work, nor one network architecture, there shouldn't be one scheduler. Instead, I think the system components should expose themselves as independent entities, and grant other system components the ability to use them under criteria. With this model, any software which can use…

You're basically describing Kubernetes and why it has become so popular

K8s is the opposite. It's proprietary, insular, not compatible with anything else, tightly coupled, not layered, not backwards compatible, etc. It has network services, logging, auth, etc, but so does literally every other system in the world, that doesn't make them all identical.

K8s is popular because it's free, has a lot of bells and whistles, and was made by Google. Otherwise nobody would use it. It's basically a larger, slightly less crappy Jenkins.

Re: Fly Kubernetes

#140

Post by verdverm »

kasey_junk wrote: post #124
verdverm wrote: post #115

Earlier quoted context omitted.

Most are pretty much actually k8s, though they tend to have different ways of handling the masters, my understanding is that it's the same k8s binaries and code What I've seen is the k8s APIs working their way into other systems like cloud functions and servers, so you can use the same Yaml across vendors and products. This is further solidifying k8s APIs as an industry standard Searching "managed kubernetes provider…

I’ve looked extensively at the documentation for gke autopilot (a system I use extensively) and haven’t found any documentation on how they orchestrate those clusters under the covers. I’ve always assumed it was a plugin to borg, not a different fleet orchestrator. Not to be overtly needy, but do you have a link to docs that contradicts that?

- https://cloud.google.com/kubernetes-engine/docs/concepts/clu...

- https://cloud.google.com/kubernetes-engine/docs/concepts/kub...

Each of these has more peer pages that are pretty good too

Post reply on HN