71–80 of 180 posts
Time will tell if embracing the complexity of Kubernetes was a good play for them or not. But, in all honesty, I'm pretty sad to see this happening, although I'm sure they had their reasons.
If they are reluctant and only do it because they have to, are they really the right vendor for managed k8s? What about them makes for a good trade-off when considering the many other vendors?
Sometimes you just want to run k8s without thinking too much about it, without having all the requirements that gcp have answers to.
Earlier quoted context omitted.
The node would be a virtual-kubelet. You can check out the virtual-kubelet GitHub repo for more info. Interestingly, there are already multiple providers of virtual-kubelet. For example, Azure AKS has virtual nodes where pods are Azure Container Instances. There’s even a Nomad provider. > So that’s what we do. When you create a cluster, we run K3s and the Virtual Kubelet on a single Fly Machine. So probably a cluster…
The diagram on https://virtual-kubelet.io/docs/architecture/ makes me wonder whether it's possible to have a k8s cluster where the nodes are all virtual kubelets backed by different cloud providers (and then perhaps schedule loads preferentially with selectors)
> But, come on: you never took us too seriously about K8s, right? What a strange way to admit they were wrong.
Is that what we did here? You get that this is just a `flyctl` feature and some Dockerfiles, right? You could have built FKS yourself by forking `flyctl`.
This is one of the biggest footguns of a tech company I've seen in the last decade. Time will tell if embracing the complexity of Kubernetes was a good play for them or not. But, in all honesty, I'm pretty sad to see this happening, although I'm sure they had their reasons.
Earlier quoted context omitted.
You wouldn’t use affinity rules anymore. The pods are scheduled on a single virtual-kubelet node, so if you use anti-affinity scheduling would fail.
How do you forbid running two instances of the same service on one node without anti affinity?
Am I understanding correctly that because they map a “Pod” to a “Fly Machine”, there’s no intermediate “Node” concept? If so, this is very attractive. When using GKS, we had to do a lot of work to get our Node utilization (the percent of resources we had reserve on a VM actually occupied by pods) to be higher than 50%. Curios what happens when you run “kubectl get nodes” - does it lie to you, or call each region one…
How is this the schedulers fault? Is this not just your resource requests being wildly off? Mapping directly to a "fly machine" just means your "fly machine" utilization will be low
Why should one use kubernetes? Or rather, at what point of an apps growth cycle does k8s become appropriate?
The real problem is that the point it becomes attractive to have something like Kubernetes is not too far from the point where Kubernetes becomes an overly-complex mess of disparate parts.
This is one of the biggest footguns of a tech company I've seen in the last decade. Time will tell if embracing the complexity of Kubernetes was a good play for them or not. But, in all honesty, I'm pretty sad to see this happening, although I'm sure they had their reasons.
At FarmLogs (yc 12) we had a pretty righteous gitops (homegrown) kube platform running dozens of microservices. We would not have been able to move as quickly as we did and roll out so many different features without it. This was back when people had just started to adopt it. Mesos was still a contender (lmao). We were polyglot too - python/clojure mixture. Heck, we even ran an ancient climate model called APSIM that was built in c#/mono, required all kinds of ancient fortran dependencies etc and it worked like a charm on kube thanks to containers. We had dedicated internal load balancers behind our VPN for raw access to services and endpoints, like "microservice.internal.farmlogs.com" (this was before istio, fabric networks, all the incredible progress that exists now)
I recall Brendan Burns asking me to write up a blog post for the Kube blog about our success story, but unfortunately was so saddled with product dev work and managing the team that I never found time for it.
I will absolutely adopt K8s again one day (very soon) but you need to know how to harness its capabilities and deploy it correctly. Build your own Heroku that fits your business. Use the Kube API directly. It's really not hard. It gets hard due to all the crap in the ecosystem (helm, yaml files). Hitting API direct means no yaml =)
I am stoked to see Fly offering this.
Am I understanding correctly that because they map a “Pod” to a “Fly Machine”, there’s no intermediate “Node” concept? If so, this is very attractive. When using GKS, we had to do a lot of work to get our Node utilization (the percent of resources we had reserve on a VM actually occupied by pods) to be higher than 50%. Curios what happens when you run “kubectl get nodes” - does it lie to you, or call each region one…