Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fly Kubernetes

fly.io

Post reply on HN
Print view View original

151–160 of 180 posts

Re: Fly Kubernetes

#151

Post by syrusakbary »

xena wrote: post #117
syrusakbary wrote: post #108

Earlier quoted context omitted.

It seems I misunderstood the article. I'm happy to read this in any case, thanks for the follow up!

fly.io employee here, it's basically an adaptor from the Kubernetes world of YAML to the fly.io world of Machines. How could we have framed it better so that it was more clear?

If the article was summarized with a TL;DR or phrase along the lines you mentioned "it's basically an adaptor from the Kubernetes world of YAML to the fly.io world of Machines" that would have made it way easier to understand.

In my opinion, the concept is not trivial to grasp by oneself (meaning: is not trivial to drive into that conclusion even after reading the article). So, being explicit first, and guide the reader through that concept along the article would have made a big difference.

Re: Fly Kubernetes

#152

Post by ossm1db »

javaunsafe2019 wrote: post #142
kuhsaft wrote: post #127

Earlier quoted context omitted.

It’s used widely in the Kubernetes world and is known as sidecars [0]. [0] https://kubernetes.io/blog/2023/08/25/native-sidecar-contain...

Widely? I lived in this helm, kubernetes, pulumi world the past 4 years and we followed the simple rule of one service/container per pod. Why add complexity where it’s not needed. Like running a dB a and a service in the same docker container - a no go for me and many.

Kubernetes official documentation states, "A Pod is a group of one or more containers, with shared storage and network resources, and a specification for how to run the containers."

Running more than one container in a pod is a fundamental concept of Kubernetes. Init containers and sidecars allow for a separation of concerns, which is essential for non-cloud-native workloads. Logging and telemetry are just a couple of features which may be designed and built into cloud-native applications, but legacy applications need this flexibility without modifying the application itself.

The fact you ran K8S for four years without it demonstrates only that it is not required by your workload -- not that it is "unnecessary complexity" or an "anti-pattern."

Re: Fly Kubernetes

#153

Post by ossm1db »

xena wrote: post #117
syrusakbary wrote: post #108

Earlier quoted context omitted.

It seems I misunderstood the article. I'm happy to read this in any case, thanks for the follow up!

fly.io employee here, it's basically an adaptor from the Kubernetes world of YAML to the fly.io world of Machines. How could we have framed it better so that it was more clear?

A more in-depth analysis of which parts of the Kubernetes spec are unsupported by this adaptor would be extremely useful in evaluating it's viability for any given use case.

Re: Fly Kubernetes

#154

Post by grossvogel »

verdverm wrote: post #2

If they are reluctant and only do it because they have to, are they really the right vendor for managed k8s? What about them makes for a good trade-off when considering the many other vendors?

I'm excited about this as a way to configure my Fly.io apps in a more declarative way. One of my biggest gripes about Fly.io is that there's a lightly documented bespoke config format to learn (fly.toml), and at the same time there's a ton of stuff you can't even do with that config file.

I love Kubernetes because the .yaml gives you have the entire story, but I'd _really_ love to get that experience w/o having to run Kubernetes. (Even in most managed k8s setups, I've found the need to run lots of non-managed things inside the cluster to make it user-friendly.)

Re: Fly Kubernetes

#155

Post by benpacker »

tptacek wrote: post #104
benpacker wrote: post #102

Earlier quoted context omitted.

If this is “free GKE autopilot” (autopilot billed at the same price as regular Fly Machine compute), then that changes the way I think about Fly’s basic compute pricing a lot. I would think they should highlight that a lot more in the product announcement!

Say more! What should we highlight more?

If I use GKE or any other standard Kubernetes offering (excluding GKE autopilot for now), if I have a variable workload and I want Node-level autoscaling, I will probably pay between 1.5x-2.5x in compute costs above what my Pod requests sum to because of difficulty with Node utilization.

It seems like with FKS, my pods will map directly to Fly Machines billing, and so there’s no compute that I’m paying for but not using

Re: Fly Kubernetes

#156

Post by benpacker »

spankalee wrote: post #144
btown wrote: post #80

Earlier quoted context omitted.

GKE Autopilot is an attractive option here if you don't want to worry about node utilization and provisioning. Effectively you have an on-demand infinitely-sized k8s cluster that scales up and down as you need new pods. Some caveats, but it's an incredible onramp if you're coming from a Heroku or similar PaaS and don't want to worry about the infrastructure side of things: Github Actions building images and deploying…

Why would you use GKE Autopilot over Cloud Run?

My understanding is that Cloud Run is not suitable for stateful workloads (databases, etc.)

Re: Fly Kubernetes

#157

Post by verdverm »

kasey_junk wrote: post #141
verdverm wrote: post #140

Earlier quoted context omitted.

- https://cloud.google.com/kubernetes-engine/docs/concepts/clu... - https://cloud.google.com/kubernetes-engine/docs/concepts/kub... Each of these has more peer pages that are pretty good too

You’ll note those documents are extremely careful in describing the control plane architecture to not promise you are running a stock k8s install. Which is why I’ve always assumed otherwise. But I’ll trust the sibling comment which suggests the only bespoke component in gke is storage well enough to leave it alone.

Maybe you are reading it differently than me, but when they refer to the same binaries, as if I managed it myself, as being the pieces they use, it definitely seems like the open source project being used.

> The control plane is the unified endpoint for your cluster. You interact with the control plane through Kubernetes API calls. The control plane runs the Kubernetes API server process (kube-apiserver) to handle API requests.

> A node runs the services necessary to support the containers that make up your cluster's workloads. These include the runtime and the Kubernetes node agent (kubelet)

Re: Fly Kubernetes

#158

Post by 0xbadcafebee »

verdverm wrote: post #143
0xbadcafebee wrote: post #139

Earlier quoted context omitted.

K8s is the opposite. It's proprietary, insular, not compatible with anything else, tightly coupled, not layered, not backwards compatible, etc. It has network services, logging, auth, etc, but so does literally every other system in the world, that doesn't make them all identical. K8s is popular because it's free, has a lot of bells and whistles, and was made by Google. Otherwise nobody would use it. It's basically a…

> K8s is the opposite. It's proprietary, insular, not compatible with anything else Not accurate, k8s is open source and every major tech company is developing or using it. There are a large number of companies building on top of it as well. - https://github.com/kubernetes (open source) - https://landscape.cncf.io/ (huge ecosystem) - https://k8s.devstats.cncf.io/d/9/companies-table?orgId=1 (count of contributions by…

Proprietary as in, relates only to itself, not compatible with anything other than itself. It's a monolith with no standards. You have to write custom software to make anything work with it. Nothing just works with k8s out of the box, because it provides no loosely coupled standard interface that remains backwards-compatible. It has an API that becomes obsolete every 9 months.

Jenkins and K8s do effectively the same thing. They're both monolithic applications (in K8s' case it's a monolith of microservices, but same difference), both have a manager/worker (formerly master/slave) architecture, both run arbitrary workloads, load secrets, store and retrieve logs from your workload, schedule them to execute on worker nodes that you configure, manage users and permissions, etc, etc. They're functionally extremely similar: distributed centralized systems designed to execute arbitrary tasks. The difference is mostly technical. Ironically, Jenkins is the more flexible of the two, with much more stable interfaces.

People in tech think in terms of cargo-cult imaginary categories, like an "orchestration system" - which isn't a computer science concept. Schedulers are, operating systems are, but "orchestrators" are not. It's a term made up to sell a product, like a configuration management engine (Terraform) or an application workload scheduler (K8s). Very different things that people use the same word ("orchestrator") for because it sounds cooler, but doesn't mean anything.

Re: Fly Kubernetes

#159

Post by kasey_junk »

verdverm wrote: post #157
kasey_junk wrote: post #141

Earlier quoted context omitted.

You’ll note those documents are extremely careful in describing the control plane architecture to not promise you are running a stock k8s install. Which is why I’ve always assumed otherwise. But I’ll trust the sibling comment which suggests the only bespoke component in gke is storage well enough to leave it alone.

Maybe you are reading it differently than me, but when they refer to the same binaries, as if I managed it myself, as being the pieces they use, it definitely seems like the open source project being used. > The control plane is the unified endpoint for your cluster. You interact with the control plane through Kubernetes API calls. The control plane runs the Kubernetes API server process (kube-apiserver) to handle AP…

We are well into the reeds of what doesn’t matter, only in that fly has given us a very under the covers look at their implementation that is hard to find with other alternatives. But as someone who has run K8s in other contexts I find the following to be pretty circumspect (not in a way that causes me concern, I’m a happy gke user)

> GKE Autopilot manages the entire underlying infrastructure of clusters, including the control plane, nodes, and all system components. If you use GKE Standard mode, GKE manages the control plane and system components, and you manage the nodes.

There is a mile of implementation detail in that. Which I’m happy for them to keep on their side of the street.

Re: Fly Kubernetes

#160

Post by harpratap »

btown wrote: post #80
benpacker wrote: post #8

Am I understanding correctly that because they map a “Pod” to a “Fly Machine”, there’s no intermediate “Node” concept? If so, this is very attractive. When using GKS, we had to do a lot of work to get our Node utilization (the percent of resources we had reserve on a VM actually occupied by pods) to be higher than 50%. Curios what happens when you run “kubectl get nodes” - does it lie to you, or call each region one…

GKE Autopilot is an attractive option here if you don't want to worry about node utilization and provisioning. Effectively you have an on-demand infinitely-sized k8s cluster that scales up and down as you need new pods. Some caveats, but it's an incredible onramp if you're coming from a Heroku or similar PaaS and don't want to worry about the infrastructure side of things: Github Actions building images and deploying…

GKE Autopilot is pretty much useless, very few cases where it actually turns out cheaper than simply using Cluster Autoscaler + Node autoprovisioning. Not only is the pricing absolutely absurd, they don't even allow normal K8s bursting behavior (requests need to be equal to limits) which means you not only end up paying more than regular K8s cluster but now also need to highly overprovision your pods
Post reply on HN