171–180 of 183 posts
The saddest part about Kubernetes is… after you set it all up, you still need a hacky deploy.sh to sed in the image tag to deploy! And pretty soon you’re back to “my dear friend you have built a Helm”. And so the configuration clock continues ticking…
The saddest part about Kubernetes is… after you set it all up, you still need a hacky deploy.sh to sed in the image tag to deploy! And pretty soon you’re back to “my dear friend you have built a Helm”. And so the configuration clock continues ticking…
Anyone remembers the GitOps thingy called flux? Weave was the company name. Git and Kubetnetes configuration cannot go hand in hand. You cannot go back in past indefinitely because cluster state might not be that reversible. If so, git is useless. And no, doesn't apply for database migrations. You can mostly run migrations backwards if each migration was written carefully.
Earlier quoted context omitted.
Yep, this is my main problem with k8s, it really feels like none of the choices have been made, it's all chose and configure components.
This, and because of that, claiming your app "runs in kubernetes" is completely meaningless. Concretely: Take your app. With one button click, or apt-get install ??? on all your machines, configure k8s. Now, run your app. The idea that this could work has been laughable for any k8s production environment I've seen, which means you can't do things like write automated tests that inject failures into the etcd control p…
If almost all your configuration is about getting Kubernetes set up, and not about your application setup inside Kubernetes, there probably isn't a point. But being able to use roughly the same config inside different Kubernetes is quite good.
Earlier quoted context omitted.
> all of those choices have been made and agreed upon Have they really? I have a few apps deployed on k8s and I feel like every time I need something, it turns out it doesn't do that and I'm into some exotic extension or plugin type ecosystem. Something as simple as service autoscaling (this was a few years ago) was an adventure into DIY. Moving from google cloud to AWS was a complete writeoff almost - just build it…
> Moving from google cloud to AWS was a complete writeoff almost - just build it again. Yep. Kubernetes is not just kubernetes when moving between clouds, it becomes a very opinionated product (for better or worse) with lots of vendor addons. Could someone that is familiar with one pick up on the other? Sure! But there are gotchas. And then kubernetes on prem adds the hardware lifecycle piece, and potential data loca…
I think this is gradually getting better. Networking with Gateways is better than with Ingress in this sense. Things like autoscaling groups need to get better, as they are (or were a couple of years ago) very bespoke.
Earlier quoted context omitted.
Til there's a security issue, right? Nginx is a big target.
The API of Ingress is not Nginx's API. The spec itself is basically a yaml schema, it's hard to have a vulnerability in that.
If you're using it after it's dead, you're at risk of further problems of this nature that aren't in the underly nginx reverse proxy but in the code wrapping it.
Earlier quoted context omitted.
The API of Ingress is not Nginx's API. The spec itself is basically a yaml schema, it's hard to have a vulnerability in that.
There have been critical vulns in nginx-ingress (the part which is deprecated) like this: https://kubernetes.io/blog/2025/03/24/ingress-nginx-cve-2025... If you're using it after it's dead, you're at risk of further problems of this nature that aren't in the underly nginx reverse proxy but in the code wrapping it.
On EKS I'm using whatever AWS has brewed up to integrate ELB/ALB, but I'll tend to trust it ... though maybe I shouldn't, given all the troubles I have with other integrations like secrets management.
IMO, Kubernetes isn't inevitable, and this seems to paint it as such. K8s is well suited to dynamically scaling a SaaS product delivered over the web. When you get outside this scenario - for example, on-prem or single node "clusters" that are running K8s just for API compatibility, it seems like either overkill or a bad choice. Even when cloud deployed, K8s mostly functions as a batteries-not-included wrapper around…
> K8s is well suited to dynamically scaling a SaaS product delivered over the web It’s well suited to other things as well, people are just in denial about some of them. “I need to run more than two containers and have a googleable way to manage their behavior” is a very common need.
Earlier quoted context omitted.
This, and because of that, claiming your app "runs in kubernetes" is completely meaningless. Concretely: Take your app. With one button click, or apt-get install ??? on all your machines, configure k8s. Now, run your app. The idea that this could work has been laughable for any k8s production environment I've seen, which means you can't do things like write automated tests that inject failures into the etcd control p…
> If your configuration is megabytes of impossible to understand YAML, and is also not portable to other environments, then what's the point? If almost all your configuration is about getting Kubernetes set up, and not about your application setup inside Kubernetes, there probably isn't a point. But being able to use roughly the same config inside different Kubernetes is quite good.
If you just tell kubectl to dump your pod configs, then load them on some other cluster, that definitely won't work.
If you use the management software that generated the pod setup somewhere else, that probably won't work either because the somewhere else is going to be missing the CSI and CNI you targeted. Even if those match, it'll be missing the CRDs. God help you if you want to run two programs on one Kubernetes, and there's a CRD versioning conflict in their two dependency sets.
Earlier quoted context omitted.
Most of the time you don't need redundancy. You need regular backups for exceptional circumstances. And k8s gives you more complexity, and more problems through more moving parts, to give you the possibility of using a feature you'll never need, and if you do start to use it it'll probably be instead of fixing performance problems downstream
Are we talking for personal projects where there are no expectations, or small startups where you don’t have much scale but you still care about down time and data loss? Personal projects are one thing, but even the smallest startup wants to be able to avoid data loss and downtime. If you are running everything on one server, how do you do kernel patches? You need to be able to move your workload to another server to…
Earlier quoted context omitted.
>It is if you stick to standard Kubernetes resources "If you stick to standard C..." No one does, that's the issue. Helm charts that only support certain cloud providers, operators and annotations that end up being platform specific, etc. >now give you default storage classes and ingresses Ingress is being deprecated, it's Gateway now! Welcome to hell, er, Kubernetes.
I use Kubernetes every day, and have worked with dozens of helm charts, and have yet to encounter cloud specific helm charts. Are these internal helm charts for your company? Obviously you can lock yourself in if you choose, but I have yet to see third party tools that assume a specific provider (unless you are using tools created BY that provider). At my previous spot, we were running dozens of clusters, with some o…
Helm charts may not support a cloud platform like Rancher, Azure, etc. or may have platform specific issues. First one I checked: https://docs.jfrog.com/installation/docs/helm-chart-requirem...
"When deploying a JFrog application on an AWS EKS cluster, the AWS EBS CSI Driver is required for dynamic volume provisioning. However, this driver is not included in the JFrog Helm Charts."
"JFrog validates compatibility with core Kubernetes distributions. Some Kubernetes vendors apply additional logic or hardening (for example, Rancher), so JFrog Platform deployment on those vendor-specific distributions might not be fully supported."