231–240 of 449 posts
I think the single biggest mistake people make with Kubernetes is implementing it too soon. Last company I worked for spent piles of time fighting K8s when a simple well implemented cluster would have done the job. It makes a lot of sense to build portable infrastructure, but you can scale a long ways with much simpler technologies.
I work on a 3-person DevOps team that just finished migrating ~20 services from GCE vms running docker-compose to GKE. It's taken us a little over a year. Partly because K8s has a steep learning curve, but also because safely transitioning services without disrupting product teams adds a lot of overhead. The investment is already yielding great returns. Developers are happy. Actual quote: "Kubernetes is the biggest q…
I'm on the DevOps side as well going through the same transition, k8s also allows insane customization, and I have some colleagues that are delaying our rollout unintentionally so they can play around with developing more tooling for deployments which is really frustrating. The k8s scene seems to be filled with constant scope creep and refactoring to get it just perfect before use. Either way, I agree the benefits far outweigh this annoyance that I've experienced. I'm so excited to work on developing tooling instead with my time.
However, I don't think we're free entirely from managing servers the old way with Chef / Puppet / Ansible, unless you're purely hosted there's still the rule of thumb you shouldn't run services that hold state in k8s. But with persistent vol's I do see that changing, though I'm not sure if everyone agree's that's a good idea.
The issue here is we don't have a kernel and compiler for distributed applications. So instead we have no choice but to expose that complexity to developers. They need to specify the IP address and port of a remote service to invoke. We've solved the IP problem in part with DNS, but now what happens when you want to migrate or load balance? Kubernetes solves this with service discovery, so you just name a service and the container orchestration engine worries about resolving that to a pod at runtime. But you still need to specify a port. We haven't yet figured out how to automate allocating ports like we managed to automate register allocation on a CPU, especially when application code itself might need to know them. We still require you to know how much storage you need and specify that in the definition of a persistent volume. Ideally, it would be as easy as it is to ask for an array of integers in a programming language. The compiler will figure out how much storage an application requires and give you that much from some pre-allocated pool you as a developer don't have to worry about.
But again, there is no such pre-allocated storage pool. There is no such compiler. There is no POSIX filesystem or memory standard for multi-machine networked systems. There's a fragmented system of vendor-locked services providing storage servers, database servers, cache servers, http servers, message queues, some more open than others. Kubernetes is an attempt to provide abstractions that make it possible to define an entire network of such individual servers declaratively and it's a noble effort. But it's complex because the underlying problem space is complex. Distributed computing is at the point in its evolution right now that single-machine computing was in around 1950 or so, when you needed to tell the program exactly where in memory to find and store a variable, exactly where on disk to fetch a block of bytes. Will it ever get to where we are now with device drivers, compilers, and kernel allocators and schedulers doing all the heavy lifting for you? This is what a Kubernetes engine is trying to be, but it's early in the game. Very early. I don't see the point in writing an article implicitly shaming the developers for trying to provide higher level abstractions that make the simple cases easier, any more than criticizing a kernel developer in 1960 for inventing virtual memory as if they're admitting that symbol to address resolution in a multi-processing system is too complex. Of course it's too complex! And we're trying to make it less complex.
Earlier quoted context omitted.
Docker Swarm is pretty much abandonware/on life support at best, so one should avoid using it for new stuff. Hashicorp's Nomad is the best choice on the complexity for features scale IMHO, and that's why i'm writing an article how great it is, how easier some things are and what's missing compared to Kubernetes.
And yet, Docker Compose is pretty popular for local development, so much so, that it's not uncommon to find a docker-compose.yml in the repositories for many open source projects. And Docker Swarm builds on that, by bridging the gap between Docker Compose and multi-server deployments, with tools like Swarmpit and Podman for easier management of it as well, much like Rancher does for Kubernetes. I agree that Docker Sw…
> Swarmpit and Podman
I actually meant Swarmpit ( https://swarmpit.io/ ) and Portainer ( https://www.portainer.io/ ).
Podman is another container runtime that acts as an alternative to Docker (even if it is not feature complete), so i misspoke.
Putting on my Asbestos Longjohns: The more I look into k8s ecosystem, the more I'm convinced that it's one of those things that suits FAANG etc, but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill. After all no one got fired for buying IBM and recommending Kubernetes. Most teams need a simpler deployment strategies that other's have succinctly mentio…
but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill There are very strong financial incentives for every individual developer and sysadmin to adopt Kubernetes, regardless of the impact it has on the organisation as a whole. In a sense this is engineering reaching the level of corporate maturity of the sales department who will optimise everything for…
Then that organization is doing a terrible job of aligning incentives. I'm guessing their pay structure isn't terribly merit-based nor high enough that people aren't constantly thinking about other jobs.
If this is about FAANG (your comment wasn't, but others were), perhaps part of this is exposing larger problems in many smaller orgs. (note: I'm ex-FAANG and happily so)
I understand their rationale. We manage thousand Kubernetes clusters and end-users can find lots and lots of creative way to shoot themselves in the foot: - I can store anything in a secret? Let's have thousands of cat images. Etcd then stops working because we have over 2GB of funny cats in the key store. - I can run a root Pod? Lets mount the docker socket and start building images with it. Oh and by the way, I nev…
One of the goal of K8S is normalization/standardization of a complex topic to better share knowledge
Earlier quoted context omitted.
but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill There are very strong financial incentives for every individual developer and sysadmin to adopt Kubernetes, regardless of the impact it has on the organisation as a whole. In a sense this is engineering reaching the level of corporate maturity of the sales department who will optimise everything for…
I'm sure there's a name to this phenomenon. Companies want stable software, and Regular Joe want better pay, but companies won't pay unless Joe starts doing crazy complex stuff that complicates things further.
Putting on my Asbestos Longjohns: The more I look into k8s ecosystem, the more I'm convinced that it's one of those things that suits FAANG etc, but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill. After all no one got fired for buying IBM and recommending Kubernetes. Most teams need a simpler deployment strategies that other's have succinctly mentio…
From my experience it's actually sold as a simpler alternative to other infra provisioning. So you end up with situations where a team deploys whatever with a helm chart, and it sets up the stuff like magic and they build on it. Then when something goes wrong they literally have no idea how to fix anything and it becomes a waking nightmare.
Also, I don't much like Go's templating syntax.
We are currently on AWS ECS which works very well for us but need to move to a GDPR/Privacy Shield compliant environment hosted in the EU. I was to look into Managed K8s but am turned off by this thread :-)