Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google admits Kubernetes container tech is too complex

theregister.com

Post reply on HN
Print view View original

431–440 of 449 posts

Re: Google admits Kubernetes container tech is too complex

#431

Post by AnHonestComment »

arcturus17 wrote: post #404
AnHonestComment wrote: post #59

Earlier quoted context omitted.

I use Docker without the network virtualization as a package manager. Docker make it easy to run the same version of code in different places and let’s things run next to each other without version conflicts. Also, I think you’re in a very small minority not to care about $720/yr increases in your hobbies.

GP was talking about projects that had revenue, and about "hiring someone" past a single instance. I replied that beyond a single instance, you can probably get away with not hiring a K8s devops person and just spinning another instance. I'm not sure you've read this whole thing right. And yes, I certainly wouldn't mind paying an additional $720 / yr for a project that had revenue; I almost certainly wouldn't want to…

GP talked about using Docker to do complex single host deployments until they needed horizontal scaling, which given max VM power, is after they can afford someone to manage it for them.

That makes me think they have multiple services of variable workload packed onto a single host, eg web server, async, and DB all on a single host via Docker.

That’s the antithesis of EB, which can only do horizontal scaling. Docker provides a way to replicate those multiple services in a deployment configuration when you want to set up a host image.

Not using Docker as an easy way to pack it all onto a single box as long as they can is just wasted expense.

Re: Google admits Kubernetes container tech is too complex

#432

Post by quadrifoliate »

ryandrake wrote: post #395
quadrifoliate wrote: post #300

Earlier quoted context omitted.

> And that they're just a complex way for engineers to keep themselves spinning their wheels and not actually working on an application...How did the world get into this situation where you need all this complexity Isn't this just labeling the knowledge that you don't have (and could probably read up on) as potentially unnecessary complexity? I mean, every time I hear about embedded, I keep hearing about byte boundar…

Fair enough--we perceive things we're ignorant about as complex. I buy that. I guess I was comparing it to how things were (or more precisely, how I perceived things) 20 years ago. You'd rack a server or two, install Linux, stick the application in /usr/local/bin, make sure Apache was set up, and you were off and running. Simple enough. It probably didn't scale to 2020-sized Internet user counts, though.

> You'd rack a server or two, install Linux, stick the application in /usr/local/bin, make sure Apache was set up, and you were off and running. Simple enough.

That sounds incredibly complex! :) Where would you find a place to rack a server? Are there minimum rates for such a place, or can you rack a single server for one month? Would you be able to rack it yourself? What if there was some problem with the PSU, and the server lost power when the data center was fine? What if the network card failed? What about if a hard drive in the server blew out?

Cloud infra in 2021 infrastructure has pretty turnkey answers to all of those (Kubernetes is not one for any of the ones I mentioned, except maybe for “what happens when any node goes down“). So yes, the complexity might be high, but so are the capabilities.

I mean, maybe you are running a mom-and-pop shop with a server rack in the basement for some reason, and have pretty daytime-specific hours. That's fine, and I still think you can do the server rack thing. Or at least that's the way I see it.

Perhaps one piece of meta-commentary here is that server hardware has not become substantially cheaper, faster, smaller, and easier to set up since 20 years ago. I can't really set up a server in my small bedroom that's capable of serving 10k concurrent users, and doesn't deafen me with its noise. Maybe if I did, all this cloud stuff might be less ubiquitous?

Re: Google admits Kubernetes container tech is too complex

#433

Post by FridgeSeal »

cies wrote: post #388
bavell wrote: post #255

Earlier quoted context omitted.

Agreed. I'm a solo technical founder and have been using k8s for all my hosting for 3+ years. It's so easy (for me) that I'm fine paying a premium for the managed service (GCP) since it saves me lots of time, my most valuable resource. I've already climbed most of the learning curve so YMMV, but as a team of one and dozens of WordPress, MySQL, and bespoke app servers, kuberenetes makes ops manageable so I can spend t…

Curious: do you have a single workload (like a WP site) that requires more than one physical computer in resources? I think that's the first thing with k8s: it all starts with an app that requires several physical nodes.

I’ve run a production kubernetes cluster that was hosting a DGraph cluster of 3 machines on its own, some ML workloads, and 4-5 products (each consisting of multiple services) and that was more than a single machine would have been able to handle.

Well _technically_, sure, we could have run a bunch of those products on a single machine, but there goes your durability and the memory overhead on some of them was quite Hugh, and properly fitting them onto a single machine would have required more optimisation and technical skills than the devs I was working with had or were inclined to do.

Re: Google admits Kubernetes container tech is too complex

#434

Post by tormeh »

dilyevsky wrote: post #88
fen4o wrote: post #13

I understand their rationale. We manage thousand Kubernetes clusters and end-users can find lots and lots of creative way to shoot themselves in the foot: - I can store anything in a secret? Let's have thousands of cat images. Etcd then stops working because we have over 2GB of funny cats in the key store. - I can run a root Pod? Lets mount the docker socket and start building images with it. Oh and by the way, I nev…

I once misconfigured iptables and locked myself out of our buildserver. Had to call lab support in a different country. Is Linux too complicated? Joyent famously took down their whole region by rebooting wrong nodes. It’s almost like running distributed networks of supercomputers at scale is hard or something...

I once did an `apt autoremove` on a custom install of CentOS handed to my team. Apt uninstalled python (and a lot more), and apt depends on python to run, so that was a bummer. The easiest way out was to reinstall the OS.

Re: Google admits Kubernetes container tech is too complex

#435

Post by nirui »

dbradstock wrote: post #383
nirui wrote: post #323

So based on the article, Autopilot is designed for GKE??? ( https://cloud.google.com/blog/products/containers-kubernetes... ) I'm learning Kubernetes and is deploying my own test cluster on my ARM-based board at this very moment, and I already spend 3 days on K3s and have to give up due to a problem ( https://github.com/k3s-io/k3s/issues/2509#issuecomment-78657... ). I must say, this is way way harder than Docker Swa…

Disclosure: I am a Googler on GKE Google actually works a huge amount with the community to simplify Kubernetes across a number of SIGs. Its always a trade of increased flexibility and options as people use it for more workloads, versus simplicity. Autopilot is just for GKE. You can use GKE on other clouds and also onprem (bare metal or VMs with Anthos)

Actually, deploy Kubernetes is not that hard after I learned the few concepts.

However, I found myself spend too much time on finding out what settings should be put into which files. I really hope there is a tool that can help users to generate and configure these settings.

I think a good example of this is `npm config edit`. When invoked, it'll open the correct config file, and list all available settings with their default values. User can then enable those settings by uncomment them as needed.

Maybe add similar functionalities into those CLI tools (kubeadm, kubectl etc) could greatly improve user-friendliness?

Re: Google admits Kubernetes container tech is too complex

#436

Post by throwdbaaway »

bedobi wrote: post #208

At a previous job, our build pipeline * Built the app (into a self container .jar, it was a JVM shop) * Put the app into a Ubuntu Docker image. This step was arguably unnecessary, but the same way Maven is used to isolate JVM dependencies ("it works on my machine"), the purpose of the Docker image was to isolate dependencies on the OS environment. * Put the Docker image onto an AWS .ami that only had Docker on it, an…

That's indeed the best approach for those who are already using a public cloud provider, to delegate the heavy lifting to ASG/MIG and ELB/NLB/GCLB/iLB, while keeping the stack nicely isolated.

At work, we have had multiple big and bigger incidents inflicted by the "share everything" nature of kubernetes. Sadly, it never occurred to the devops team that something like this is possible. Same goes for the HN crowds here who just assume that a container scheduler is a must, be it kubernetes or nomad or docker swarm.

I blame this on Google Cloud brainwashing. Let's see how long it will take for Thomas Kurian to sunset https://cloud.google.com/compute/docs/containers/deploying-c...

Re: Google admits Kubernetes container tech is too complex

#437

Post by fy20 »

diarrhea wrote: post #153
fy20 wrote: post #65

Earlier quoted context omitted.

I use Swarm at home (only on a single node, because it turns out 3 are overkill for my needs) and it's been running great for 6 months so far. Before that I tried various incarnations of k8s and eventually they'd just destroy themselves up and require a rebuild (the main issue was persistent storage). My only complaint with Swarm is there isn't an easy way to expose containers directly on the network (like host netwo…

Mind sharing a bit more on your home setup? Asking because I never saw the point in multiple container replicas for simple self-hosted stuff. One container each has served me well so far [0] (Nextcloud, Bitwarden, GitLab), and if they crash, they just get restarted. Multiple containers increase throughput, supporting more users, is that it? It just sounds nightmarish in regards to storage and parallel, conflicting wr…

I do have just one container of each thing, but I was originally planning to have multiple nodes. I have 3x ThinkCenter Tiny M73 (with the Pentium CPU) and thought one would be a bit underpowered for everything I wanted to run (it was for k8s), so was planning to distribute services automatically across the swarm. One node is more than enough though, so I'd actually be fine with just docker-compose, but splitting everything up into separate 'services' is nice.

Re: Google admits Kubernetes container tech is too complex

#438

Post by fy20 »

lub wrote: post #78
fy20 wrote: post #65

Earlier quoted context omitted.

I use Swarm at home (only on a single node, because it turns out 3 are overkill for my needs) and it's been running great for 6 months so far. Before that I tried various incarnations of k8s and eventually they'd just destroy themselves up and require a rebuild (the main issue was persistent storage). My only complaint with Swarm is there isn't an easy way to expose containers directly on the network (like host netwo…

Did you try this? networks: - host https://docs.docker.com/network/host/

Maybe it has changed since I built it, but I wasn't able to get this working with Swarm services. I had to convert them to docker-compose to make it work. The docs suggest it should work with Swarm mode though, so maybe I need to try again.

Ideally I'd like to give each service its own IP on the network, which was possible with how I had k8s setup.

Re: Google admits Kubernetes container tech is too complex

#439

Post by MaxBarraclough »

__jem wrote: post #322
cameronh90 wrote: post #279

Earlier quoted context omitted.

By that logic, processes are arguably virtualisation too. They do after all use virtual memory. Threads, processes and containers exist on a continuum.

The key is that "containers" don't actually exist -- they're just processes running under a variety of different namespaces.

It's true that Docker isn't a first-class abstraction at the level of the Linux kernel, but BSD has jails, and Solaris has Zones. This is important in some respects, but I don't see that it informs things here. Containers are still 'a thing' regardless of how they're implemented.

Re: Google admits Kubernetes container tech is too complex

#440

Post by dilyevsky »

diarrhea wrote: post #148
FlyingSnake wrote: post #109

Putting on my Asbestos Longjohns: The more I look into k8s ecosystem, the more I'm convinced that it's one of those things that suits FAANG etc, but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill. After all no one got fired for buying IBM and recommending Kubernetes. Most teams need a simpler deployment strategies that other's have succinctly mentio…

A friend of mine is a contributor to k8s itself, and of course, this all comes incredibly easy to them. Following their recommendation, I gave it a shot for my single-person, single-node (!) homelab, all without using MicroK8s, k3s or similar. After a week of almost full-time work, I threw in the towel. Admittedly, I also had to learn concepts like reverse proxies alongside, too, so I was by no means well-equipped to…

Wait so running a kubeadm init and then removing a master taint took you whole week to figure out? How long ago was that?
Post reply on HN