Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google admits Kubernetes container tech is too complex

theregister.com

Post reply on HN
Print view View original

221–230 of 449 posts

Re: Google admits Kubernetes container tech is too complex

#221

Post by anchochilis »

I work on a 3-person DevOps team that just finished migrating ~20 services from GCE vms running docker-compose to GKE.

It's taken us a little over a year. Partly because K8s has a steep learning curve, but also because safely transitioning services without disrupting product teams adds a lot of overhead.

The investment is already yielding great returns. Developers are happy. Actual quote: "Kubernetes is the biggest quality-of-life improvement I've experienced in my career."

The same is true for me as a DevOps engineer. I never have to write a script to orchestrate a rolling deployment ever again. I'll never have to touch Puppet or wrangle OS upgrades. I no longer need to use Terraform to scale out a service.

Now that we've completed this transition, one of my goals for the next year is to get the team to a place where we spend 75% of our time on higher-level projects -- working on developer tools and Kubernetes extensions and setting site reliability standards, instead of the firefighting and operational work that has traditionally taken up so much of our time. Couldn't be happier about this change.

... That is until management decides we're overhead and fires us to free up budget for feature developers. ;)

Re: Google admits Kubernetes container tech is too complex

#222

Post by vitus »

pjmlp wrote: post #210
fendy3002 wrote: post #205

Earlier quoted context omitted.

I suppose your apps ran on windows. It isn't a problem (or at least it's a smaller problem) with windows ecosystem, especially enterprise one, since usually you're using same version and installation steps handled by infra with AD. Even without AD usually the installer and windows version is same, and Microsoft usually great at backwards compatibility. But it's not the case in linux, or at least non-enterprise / ldap…

Back when I was doing big boys UNIX, the respective package managers took care of everything. Which we later replicated in Red-Hat with RPM. Bare bones OS install + bunch of OS packages => done. And in what concerns containers I was working with HP-UX Vault in 1999.

Agreed to an extent. Even outside RHEL, you have yum for RPM-based distros, Aptitude for deb-based distros, pacman for Arch, etc.

You need to start with the same base operating system, and you need to make sure that you pull in the same versions of packages in case there are backwards-incompatible bugs, or version bumps such that the dynamically-loaded library is no longer detected (hence the common albeit dangerous workaround of "add a symlink").

(If you're using rpm directly, then you need to bundle the actual packages that you're installing, or point to specific packages that you're confident won't change. And at that point, what's the difference between your approach and Docker?)

The challenge that I believe Docker solves (or, at least, attempts to) is environment reproducibility: without it, you have dependency hell.

Re: Google admits Kubernetes container tech is too complex

#224

Post by JeremyNT »

runeks wrote: post #119
fen4o wrote: post #13

I understand their rationale. We manage thousand Kubernetes clusters and end-users can find lots and lots of creative way to shoot themselves in the foot: - I can store anything in a secret? Let's have thousands of cat images. Etcd then stops working because we have over 2GB of funny cats in the key store. - I can run a root Pod? Lets mount the docker socket and start building images with it. Oh and by the way, I nev…

> I can store anything in a secret? Let's have thousands of cat images. Why would someone want to store non-secret information as a secret?

Because mutable persistence in Kubernetes can be super annoying to manage and people might grasp for whatever lifeline they can find.

If you have a managed object store or even relational database outside of k8s, the thought of storing arbitrary data in secrets probably doesn't come to mind. But if your enterprise spools up a cluster and tells you to use nfs PVCs with no other storage solution, suddenly you might start getting creative.

Re: Google admits Kubernetes container tech is too complex

#225

Post by fctorial »

oauea wrote: post #51
ryandrake wrote: post #43

Earlier quoted context omitted.

My background is in embedded, so I admittedly know extremely little about web development, but whenever I'm curious and sit down to read about microservices and containers and orchestration and all that stuff, my mind starts to numb and I can't help but conclude that 99% of companies that use it probably don't need to. And that they're just a complex way for engineers to keep themselves spinning their wheels and not…

Yeah, everyone feels that way until they ran into several issues that suddenly make containerization look like a great idea.

examples?

Re: Google admits Kubernetes container tech is too complex

#226

Post by fctorial »

aequitas wrote: post #108
ryandrake wrote: post #43

Earlier quoted context omitted.

My background is in embedded, so I admittedly know extremely little about web development, but whenever I'm curious and sit down to read about microservices and containers and orchestration and all that stuff, my mind starts to numb and I can't help but conclude that 99% of companies that use it probably don't need to. And that they're just a complex way for engineers to keep themselves spinning their wheels and not…

> How did the world get into this situation where you need all this complexity just to deploy and configure not even an application, but part of an application? Amazing! Because everyone else is doing it. Things have gotten so complex that we need complex solutions, but w're afraid to make and own the solution ourselves because of cost, focus on core business or recruitment/knowledge concerns. So we need to find the…

There's an xkcd for that.

Re: Google admits Kubernetes container tech is too complex

#227

Post by fennecfoxen »

bob1029 wrote: post #176

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

I get where you're coming from, but it turns out that plopping the entire VM into a zipfile ends up being a good way to have the kind of reproducibility that makes your operations sane. Do you pin specific versions of dependencies for your install scripts? Might as well pin the whole image. It's like 88% of the benefit of reproducible builds, at 3% of the cost, and that's not nothing.

Mind you, that's still Docker, though, not Kubernetes.

> If one of your consultants or developers tells you they can make your application faster by adding a bunch of additional computers, you are almost certainly getting taken for a ride.

Eh. There's a redundancy play in there somewhere too, if you know how to pull it off. (Big if.)

Re: Google admits Kubernetes container tech is too complex

#228

Post by cactus2093 »

pantelisk wrote: post #184
bob1029 wrote: post #176

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

I agree with you. However, after spending years and years trying to compile software that came with cryptic install instructions. Or have the author insist that since it works on their machine I 'm just doing something stupid. Docker was largely able to fix that. It's a somewhat odd solution for a too common problem, but any solution is still better than dealing with such an annoying problem. (source: made docker the…

Completely agree. The whole compile chain for most software and reliance on linked libraries, implicit dependencies like locale settings changing behavior, basically decades of weird accidents and hacks to get around memory and disk size limits, can be a nightmare to deal with. If using slow dynamic languages, or modern frontend bunglers, all the implicit c extension compilations and dependencies can still be a pain.

The list goes on and on, it’s bizarre to me to think of this as the true, good way of doing software and think of docker as lazy. Docker certainly has its own problems too, but does a decent job at encapsulating the decades of craziness we’ve come to heavily rely on. And it lets you test these things alongside your own software when updating versions and be sure you run the same thing in production.

If docker isn’t your preferred solution to these problems that’s fine, but I don’t get why it’s so popular on HN to pretend that docker is literally useless and nobody in their right mind would ever use it except to pad their resume with buzzwords.

Re: Google admits Kubernetes container tech is too complex

#229

Post by yebyen »

jononomo wrote: post #189
YawningAngel wrote: post #173

Earlier quoted context omitted.

That would do nothing to help the 99% of users who don't use BEAM VM languages

They'd have a reason to start using BEAM VM languages, though.

To a person who isn't using Erlang or ASP.net, suggesting that we should use either of those language packages and it will solve any of our problems without creating a thousand new ones sounds equally non-starterish to me.

To add a counter-example, I have lots of Ruby experience and I've just joined a Go team. I won't tell them to use Ruby, I will just do it where it makes sense and saves us time. (And then we'll have two problems... enter "limiting blast radius")

Point of my counter-example is, I'm extremely skeptical that all the world's problems can be solved by adopting a new mono-culture, whatever it is. There are 100% always gonna be some problems that are better solved in a different language. PHP is the best way to run Wordpress, for example (ok, so it's the only way to run Wordpress, but you get the idea... "Wordpress is the best way to..."), but I've been in high-functioning IT organizations that won't touch that with a ten foot pole, because "it's another language to support, and PHP is icky."

We also got rid of a perfectly fine Wiki in favor of centralized Knowledge-base software for similar reason. "Better to just have one KB. We don't need to be hosting another thing." So the chances of moving everything over to BEAM VM are next to nil, unless you are a product-focused company with just one product, or happen to have an absolute champion leading the effort to migrate all the things. For all the other things, you need to have a consistent answer too.

No tool is one-size-fits-all. Where Kubernetes shines most is under any environment that isn't running a single monolith or building a software monoculture and/or can't manage that for whatever reason (because those are all basic use cases that are frankly easy enough to manage without adding on top the additional complexity of Kubernetes; don't need it, don't use it!) IMHO, diversity in infrastructure is a plus though, and Kubernetes is a technology that it turns out enables this.

Re: Google admits Kubernetes container tech is too complex

#230

Post by bestinterest »

yonixw wrote: post #194
bob1029 wrote: post #176

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

> Horizonal scalability is simply a band-aid for poor engineering in most (not all) applications. Maybe if your app handles < 10k concurrent connections. Otherwise it is the most cost efficient solution and exists because it solves the scaling problem in the best way as of today.

Does anyone know of any raw stats at what one beefy server can handle with a typical HTTP CRUD app? <10k doesn't seem right?
Post reply on HN