Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google admits Kubernetes container tech is too complex

theregister.com

Post reply on HN
Print view View original

251–260 of 449 posts

Re: Google admits Kubernetes container tech is too complex

#251

Post by alephu5 »

MaxBarraclough wrote: post #215
bob1029 wrote: post #176

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

> Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). At the risk of nitpicking, docker images aren't the equivalent of VM images, as they don't include a kernel.

I agree with the other commentor, this is an important distinction. It has no hypervisor and is really just a normal process using standard kernel features: cgroups (resource limiting) and namespaces (resource isolation). It's really not so different to chroot.

Re: Google admits Kubernetes container tech is too complex

#252

Post by 2sk21 »

dilyevsky wrote: post #90
trinovantes wrote: post #36

Kubernetes has to be most complex software I've ever tried to learn. I eventually gave up and decided to stick with simple single machine docker-compose deployments. I figure by the time any of my personal projects actually need to scale beyond 1 machine, I'd probably have enough revenue that I can afford to hire someone else to worry about it.

Long time ago I worked on HA setups for telecom (Wimax/LTE) equipment. Kubernetes is complicated but has nothing on those systems. Just to give you some idea - https://www.metaswitch.com/hs-fs/hubfs/Blogs/3gpp-ts-23-228-... (doesn’t even cover everything)

The very term "HA" still gives me nightmares. It can be very hard to get HA to work correctly. Many years ago, I worked in a startup and one of our main offerings was an HA network device. It was unbelievably finicky to get it to work in the first place and even harder to update the software on an HA cluster.

Re: Google admits Kubernetes container tech is too complex

#253

Post by rileymat2 »

AndyMcConachie wrote: post #196
bob1029 wrote: post #176

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

Containers solve the dependency problem by simply pretending it doesn't exist. I used to do UNIX integration work in the late 1990's early 2000's and containers weren't really a thing. So you had to make sure libs from one program didn't crap on another program. And developers had to be conscious of what dependencies they included in their code. Nowadays they don't have to care as much because of containers. Every pr…

Why is this a better solution?

Separating the dependencies between programs allows you to test and release independently to allow incremental upgrades. IMO that is better.

Re: Google admits Kubernetes container tech is too complex

#254

Post by anchochilis »

z9e wrote: post #233
anchochilis wrote: post #221

I work on a 3-person DevOps team that just finished migrating ~20 services from GCE vms running docker-compose to GKE. It's taken us a little over a year. Partly because K8s has a steep learning curve, but also because safely transitioning services without disrupting product teams adds a lot of overhead. The investment is already yielding great returns. Developers are happy. Actual quote: "Kubernetes is the biggest q…

... Until they hire you back because entropy exists :P I'm on the DevOps side as well going through the same transition, k8s also allows insane customization, and I have some colleagues that are delaying our rollout unintentionally so they can play around with developing more tooling for deployments which is really frustrating. The k8s scene seems to be filled with constant scope creep and refactoring to get it just…

Ha, are you me? I really pushed for us to follow the "change-as-little-as-possible and ship to prod quickly" route. Prod is where things get hard, and it's better to find out what's hard sooner rather than later.

We are running a handful of stateful services in K8s (things like MongoDB for which GCP doesn't have a compelling and affordable managed offering). It's definitely more complex than transitioning a stateless service, but so far our experiences with StatefulSets and PersistentVolumes have been good. And this allows us to sunset Puppet/OS management completely. I should note that we _are_ being extremely careful about backups. We also run each stateful service in a dedicated node pool for isolation. Who knows, maybe a year from now we'll be shaking our heads and saying "that was a TERRIBLE idea" but for now, so far so good.

We're running on GKE, so lots of things that would be hard in on-prem environments (ingress, networking, storage) are easy.

Re: Google admits Kubernetes container tech is too complex

#255

Post by bavell »

FridgeSeal wrote: post #207
cies wrote: post #128

Earlier quoted context omitted.

I once have been told that development teams smaller than 20 developers have no business in using k8s, due to the complexity to brings. If something as essential as the infra so complex it is not readily understood by everyone on the team, a few (more than one) team members need to become the experts on the matter. For small teams this is simply not worth it.

As part of a small team currently using Kubernetes, I suspect it’s more how you use it - the tools and ecosystem have matured immensely in the last couple of years since I’ve first started using it. I don’t think it suits all teams and use cases, but for us it’s absolutely fantastic and without going down the rabbit-hole of cloud-provider specific tools and recreating half the issues it solves, I’m not super sure wha…

Agreed. I'm a solo technical founder and have been using k8s for all my hosting for 3+ years. It's so easy (for me) that I'm fine paying a premium for the managed service (GCP) since it saves me lots of time, my most valuable resource.

I've already climbed most of the learning curve so YMMV, but as a team of one and dozens of WordPress, MySQL, and bespoke app servers, kuberenetes makes ops manageable so I can spend time on things that really matter.

Deploying new web apps is trivial, declarative manifests are easy to reason about, TLS certs are issued and renewed automatically (cert-manager), backups are cheap and reliable (daily GCP snapshots), making changes to the cluster via declarative terraform is a breeze, etc etc. No way I could manage all the ops without leaning so heavily on the core foundation provided by k8s.

Re: Google admits Kubernetes container tech is too complex

#256

Post by [deleted] »

rbanffy wrote: post #212
bob1029 wrote: post #176

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

> Horizonal scalability is simply a band-aid for poor engineering And don't even get me started on having instances labeled "large" that have less memory and CPU capacity than my personal backup laptop (currently on loan to my 8yo for COVID reasons)...

[deleted]

Re: Google admits Kubernetes container tech is too complex

#257

Post by yonixw »

rbanffy wrote: post #217
yonixw wrote: post #194

Earlier quoted context omitted.

> Horizonal scalability is simply a band-aid for poor engineering in most (not all) applications. Maybe if your app handles < 10k concurrent connections. Otherwise it is the most cost efficient solution and exists because it solves the scaling problem in the best way as of today.

When you approach the limits of what your kernel can handle, then it may be time to split your workload across boxes or to carve smaller boxes out of your metal (and probably directly attaching NICs to the VMs to the host OS doesn't have to deal with them). Making your workload horizontally scalable is always a sound engineering choice. But... Splitting a horizontally scalable workload across a dozen virtual servers…

> When you approach the limits of what your kernel can handle

Even before. If you want low latency. And banks handle more than 10K concurrent every day.

Cost example: https://pt.slideshare.net/markmyers106/vertical-vs-horizonta...

Re: Google admits Kubernetes container tech is too complex

#258

Post by m463 »

bob1029 wrote: post #176

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

Personally I think there is beauty to one text file that defines a reproducible build environment. Carefully controlled dependencies are quite nice.

That said, I haven't really used docker as a runtime container.

Re: Google admits Kubernetes container tech is too complex

#259

Post by Ianvdl »

anchochilis wrote: post #221

I work on a 3-person DevOps team that just finished migrating ~20 services from GCE vms running docker-compose to GKE. It's taken us a little over a year. Partly because K8s has a steep learning curve, but also because safely transitioning services without disrupting product teams adds a lot of overhead. The investment is already yielding great returns. Developers are happy. Actual quote: "Kubernetes is the biggest q…

As someone with limited experience with containers, how does K8s allow you to move away from things like Puppet for configuration management? Does it offer some substitute that alleviates the need for something like Puppet or Ansible?

Re: Google admits Kubernetes container tech is too complex

#260

Post by bavell »

pram wrote: post #200
FlyingSnake wrote: post #109

Putting on my Asbestos Longjohns: The more I look into k8s ecosystem, the more I'm convinced that it's one of those things that suits FAANG etc, but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill. After all no one got fired for buying IBM and recommending Kubernetes. Most teams need a simpler deployment strategies that other's have succinctly mentio…

From my experience it's actually sold as a simpler alternative to other infra provisioning. So you end up with situations where a team deploys whatever with a helm chart, and it sets up the stuff like magic and they build on it. Then when something goes wrong they literally have no idea how to fix anything and it becomes a waking nightmare.

As a longtime and frequent user of k8s, I stay away from helm charts. I tried them out when they first got popular but I found they introduced more friction than they solved on the whole.

Not every addon/tool for the k8s ecosystem is worth it. I also don't bother with the ever-growing list of service meshes... not enough value to me for the overhead.

K8s is definitely the simpler alternative for me but there is still a lot of essential complexity in k8s due to the nature of the problems it's trying to solve. Mostly I like building on top of a solid foundation of standardized k8s API objects (pods, services, volumes, etc).

Tldr; Bring in only the add-ons and tools you really need so you don't add more complexity than necessary. Don't get swept up in the hype and marketing from other devs and cloud vendors.

Post reply on HN