I have concluded that most (perhaps all) cloud offerings of Kubernetes clusters are a prime example of "enshitification". You take something that should be easy and trivial, then add layers of complexity and vendor-specific components, slightly different configuration options, lots of plug-ins you probably won't ever need (but you need to know which ones you will, and how to decide which is which), and so on. The result is that, when you are finished, you made something that runs only on the specific provider you selected.
I Didn't Need Kubernetes, and You Probably Don't Either
321–330 of 436 posts
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#322We dabbled with Cloud Run and Cloud Functions (which as of v2 are just a thin layer over Cloud Run anyways). While they worked fine for HTTP workloads, we wanted to use them to consume from Pub/Sub and unfortunately the "EventArc" integrations are all HTTP-push based. This means there's no back pressure, so if you want the subscription to buffer incoming requests while your daemons work away there's no graceful way t…
> The push subscription will just ramp up attempting to DoS your Cloud Run service
Interesting. My assumption would be that Cloud Run should quickly* spin up more containers to handle the spike and then spin them down again. So there would be no need for back pressure? Guess it depends on the scale? How big of a spike are we talking about? :)
*Let's say a few seconds
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#323Earlier quoted context omitted.
So, let's say you want to deploy server instances. Let's keep it simple and say you want to have 2 instances running. You want to have zero-downtime-deployment. And you want to have these 2 instances be able to access configuration (that contains secrets). You want load balancing, with the option to integrate an external load balancer. And, last, you want to be able to run this setup both locally and also on at least…
Cloud Run. Did you read the article? Migrating to another cloud should be quite easy. There are many PaaS solutions. The hard parts will be things like migrating the data, make sure there's no downtime AND no drift/diff in the underlying data when some clients write to Cloud-A and some write to CLoud-B, etc. But k8 do not fix these problems, so..
Came here to say the same thing: PaaS. Intriguing that none of the other 12 sibling comments mention this… each in their bubble I guess (including me). We use Azure App Service at my day job and it just works. Not multi-cloud obviously, but the other stuff: zero downtime deploys, scale-out with load balancing… and not having to handle OS updates etc. And containers are optional, you can just drop your binaries and it runs.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#324Interesting that the mania for over-investment in devops is beginning to abate. Here on Hacker News I was a steady critic of both Docker and Kubernetes, going to at least 2017, but most of these posts were unpopular. I have to go back to 2019 to find one that sparked a conversation: https://news.ycombinator.com/item?id=20371961 The stuff I posted about Kubernetes did not draw a conversation, but I was simply document…
It's just the hype moving on. Every generation has to make similar mistakes again and again. I am sure if we had the opportunity and the hype was there we would've used k8s in 2005 as well. The same thing is true for e.g. JavaScript on the frontend. I am currently migrating a project from React to HTMX. Suddenly there is no build step anymore. Some people were like: "That's possible?" Yes, yes it is and it turns out…
[dead]
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#325Earlier quoted context omitted.
until, as happened to us, you're in the middle of an upgrade cycle, with a mix of red-hat 6 and red-hat 8 servers, and ansible decide to require support for the latest available version of python on red-hat 8, which isn't available on red-hat 6, so we have no way of using ansible to manage both sets of servers. The python ecosystem is a cancer.
Well, you were free to install a version of Python3 on the CentOS6 machines, that's what we ended up doing and using for Ansible. Python 2.6 support of Ansible was a bad lie, multiple things broke already. 10 years of support without acknowledging changes of ecosystem just don't work.
we did, but then most ansible modules still didn't work on the system. They advertise the no-agent thing and how it does everything over ssh, and instead require python to be installed on all your servers because it generates python code and runs it on the remote machine. And somemodules require specific versions sometimes.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#326Earlier quoted context omitted.
I find it always interesting and weird to read so dimetrical points to kubernetes. 1. Cloud for me is a lot better than what we had before: Before i had to create a ticket for our internal it department, have huge cross-charges (like 500$ for a server, instead of 50), had to wait for a few weeks and than get lectured that installing basic default tools on that suse based server would take a week and add additional cr…
> shitty shell scripts, ansible setup and co, i only write a little bit of yaml Why are the shell scripts shity but the yaml not? When I look at those yaml files I always throw up just a little :P Also, have you tried Cloud Run?
Yaml is declarative, you tell k8s what you want and how it looks.
For shell scripts, try proper error handling. You start doing some catch hooks, you have issues cehcking error codes of different tools, debugging is hard too.
In one infra project we swtiched from shell scripts to golang just to have a lot more control/stability of our scripts.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#327Earlier quoted context omitted.
I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…
I hardly know the first thing about Kubernetes or cloud, so maybe you can help explain something to me: There's another Kubernetes post on the front page of HN at the moment, where they complain it's too complex and they had to stop using it. The comments are really laying into the article author because they used almost 50 clusters. Of course they were having trouble, the comments say, if you introduce that much com…
Sounds like it's their primary job is to manage clusters for others, which ofc is different from trying to manage your primary service, that you deployed as 50 microservices in individual clusters (didn't read the other article)
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#328Earlier quoted context omitted.
I wouldn't throw "cloud shit" in the same bucket as Kubernetes. Professional cloud services are mostly great, but super expensive. Kubernetes is interesting, because it basically takes everything you know and sort of pushes it down the stack (or up, depending on your viewpoint). To some extend I get the impression that the idea was: Wouldn't it be great if we took all the infrastructure stuff, and just merged the who…
What do you think its niche is?
Very large "single product" services, the focus being on "very large". It would also be relevant in the cases where your product is made up of a large number of micro-services, though if you have more than 50 services to support your main product I'd be interested in knowing why you have that many services. There might be completely sane reasons, but it is a special case.
Mostly a question of scale to me, I'd guess that the majority (80-90%) of people running Kubernetes don't have large enough scale that it makes sense to take on the extra complexity. Most Kubernetes installations I've seen runs on VMs, three for control plane and 2 - 5 for worker node and I don't think the extra layer is a good trade off for a "better" deployment tool.
If you do use Kubernetes as a deployment tool, then I can certainly understand that. It is a reasonably well-known, and somewhat standardised interface and there's not a lot of good alternatives for VMs and bare metal. Personally I'd just much rather see better deployment tools being developed, rather than just picking Kubernetes because Helm charts are a thing.
You'd also need to have a rather dynamic workload, in the sense that some of your services is need a lot of capacity at one point in time, while other need the capacity at another time. If you have constant load, then why?
It's like Oracle's Exadata servers, it's a product that has its place, but the list of potential buyers isn't that long.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#329Earlier quoted context omitted.
Having done both: running a small Kubernetes cluster is simpler than managing a bunch of systemd files.
Yeah this is my impression as well which makes me not understand the k8s hate.
The complexity of k8s comes the moment you need to hold state of some kind. Now instead of one systemd entry, we have to worry about persistent volume claims and other such nonsense. When you are doing things that are completely stateless, it's simpler than systemd.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#330Earlier quoted context omitted.
> shitty shell scripts, ansible setup and co, i only write a little bit of yaml Why are the shell scripts shity but the yaml not? When I look at those yaml files I always throw up just a little :P Also, have you tried Cloud Run?
Yaml is declarative, you tell k8s what you want and how it looks. For shell scripts, try proper error handling. You start doing some catch hooks, you have issues cehcking error codes of different tools, debugging is hard too. In one infra project we swtiched from shell scripts to golang just to have a lot more control/stability of our scripts.
YAML is not declarative, it's a format. Well, according to Wikipedia it's a "data serialization language". IMO It's also bad choice for this, and those files become unreadable and hard to work with.
Agree that shell scripts are also hard to work with, especially if you did not write them yourselves. I guess it's a combo of the language features of, say bash, and that no one who writes them really know bash. I mean, at all. Including me.
Declarative is nice, but also have pros and cons. And, it's of course many ways to achieve this if that's a priority.
Usually, what you really want is: Low time to replicate AND no data loss if a server blows up. But this also have to extend to, say, the k8 cluster. And, again, many ways to achieve this.
The article does not call for Ansible setups and shell scripts though.
Cloud Run uses YAML btw. One of the things I personally don't like about it