Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Kubernetes is so complex

cloudplane.org

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Why Kubernetes is so complex

#21

Post by democracy »

MuffinFlavored wrote: post #5
vaidhy wrote: post #4

I think the answer is hidden in the last paragraph. It was created by Google engineers to solve Google scale problems where they run maybe hunderds of thousands of services with a dedicated SRE team and massive tooling. For a small company, give me a way to run one container in an autoscable way and we can go from there.

> For a small company, give me a way to run one container in an autoscable way and we can go from there. What's the best way you've found to do that?

Google App Engine https://cloud.google.com/appengine

Re: Why Kubernetes is so complex

#22

Post by nuker »

theideaofcoffee wrote: post #10
shudza wrote: post #8

Earlier quoted context omitted.

Which is why kubernetes got adopted - everyone pretends they are Google, having Google problems, when they, in fact, don't.

This is a heretical idea in some circles. Utter it and expect to be ostracized. In my case it’s cost a few job offers when I’ve suggested that a few python processes running under supervisor (e.g.) would be more than sufficient. It has its place but that point lies at a stage of development and sophistication far beyond where most cargo-cult users are currently.

This is a heretical idea in *most circles. Same experience with job offers!

Re: Why Kubernetes is so complex

#23

Post by nuker »

rufius wrote: post #7
MuffinFlavored wrote: post #5

Earlier quoted context omitted.

> For a small company, give me a way to run one container in an autoscable way and we can go from there. What's the best way you've found to do that?

ECS/Fargate for long lived things. Lambdas for short lived things.

That plus fleet of ec2 for real heavy loads.

Re: Why Kubernetes is so complex

#24

Post by nuker »

vaidhy wrote: post #20
rufius wrote: post #7

Earlier quoted context omitted.

ECS/Fargate for long lived things. Lambdas for short lived things.

+1 for Fargate and Lambda. If you do not like Fargate, you can also run containers in EC2 for a bit more control.

Nah, scaling will be hard. I came to realisation that I don’t want any clusters, let them manage it :)

Re: Why Kubernetes is so complex

#25

Post by techdragon »

jacooper wrote: post #16

What about Nomad?, I hear it isn't as complex as K8s, but still offers the same capabilities. Also in what way is Swarm is abandoned?, I mean if it works fine, and is still supported in Docker-CE than its still OK to use it, at least in small businesses and hobbyist use cases where Swarm's simplicity are attractive.

I recently looked at Nomad, but ironically for a "less complex" piece of software, I couldn't be sure it would scale down to my use case, I ended up with K3S as it would run on the small hardware nodes i needed it to run on.

I would have preferred Nomad but the resource requirements are pretty high for the "control server" component. https://www.nomadproject.io/docs/install/production/requirem...

  Nomad servers may need to be run on large machine instances. 
  We suggest having between 4-8+ cores, 16-32 GB+ of memory,
  40-80 GB+ of fast disk and significant network bandwidth. 
  The core count and network recommendations are to ensure
  high throughput as Nomad heavily relies on network communication
  and as the Servers are managing all the nodes in the region and
  performing scheduling. The memory and disk requirements are due to
  the fact that Nomad stores all state in memory and will store two
  snapshots of this data onto disk, which causes high IO in busy
  clusters with lots of writes.
Obviously This is not going to fit on a group of Raspberry Pi's or other SBC computer nodes you can solar power out in a field.

Re: Why Kubernetes is so complex

#26

Post by Cwizard »

Could someone explain to me in what why Kubernetes is complex and what alternatives are simpler? I’ve worked on non-k8s systems before and in my experience they all hang together with custom bash/python code which although line-for-line is ‘simpler’ it makes it harder to onboard new people and is less robust (excluding very simple deployments)

K8s is very modular in my experience so if you don’t need something you can easily ignore it and not pay a complexity cost. Nomad does not seem much simpler to me (especially because you basically have to pair it with Consul and Vault)

I am genuinely curious.

Re: Why Kubernetes is so complex

#27

Post by mradek »

shudza wrote: post #8
vaidhy wrote: post #4

I think the answer is hidden in the last paragraph. It was created by Google engineers to solve Google scale problems where they run maybe hunderds of thousands of services with a dedicated SRE team and massive tooling. For a small company, give me a way to run one container in an autoscable way and we can go from there.

Which is why kubernetes got adopted - everyone pretends they are Google, having Google problems, when they, in fact, don't.

I think it’s also probably because it gives people a “predictable” way to do things.

That being said I sometimes ask myself why we can’t constantly think KISS and YAGNI. Like, do we really need this level of abstraction and complexity? I’ve been “working” with k8s and I would probably fail any interview on it because I feel like I’m always googling my way through issues. I don’t even care anymore because I know for my own purposes outside of work, I keep my code and systems stupid simple.

And maybe this sounds cringey to some but I’m happy to write a few scripts on my own to handle deployments without needing to break my software into a thousand pieces. Single responsibility code using a few languages that are best suited for the task at hand (in my case it’s mostly node, elixir, go) that’s easy to break apart and ship separately is so nice. Why can’t we do the same at work?

Oh well, I’ll collect my check 2x a month thanks.

Re: Why Kubernetes is so complex

#28

Post by perfecthjrjth »

vaidhy wrote: post #4

I think the answer is hidden in the last paragraph. It was created by Google engineers to solve Google scale problems where they run maybe hunderds of thousands of services with a dedicated SRE team and massive tooling. For a small company, give me a way to run one container in an autoscable way and we can go from there.

Large companies are done with huge k8s clusters. Now every app team and their SRE team rolls out their own k8s cluster. I am not sure whether these companies are solving Google scale problems.

Re: Why Kubernetes is so complex

#29

Post by cb22 »

techdragon wrote: post #25
jacooper wrote: post #16

What about Nomad?, I hear it isn't as complex as K8s, but still offers the same capabilities. Also in what way is Swarm is abandoned?, I mean if it works fine, and is still supported in Docker-CE than its still OK to use it, at least in small businesses and hobbyist use cases where Swarm's simplicity are attractive.

I recently looked at Nomad, but ironically for a "less complex" piece of software, I couldn't be sure it would scale down to my use case, I ended up with K3S as it would run on the small hardware nodes i needed it to run on. I would have preferred Nomad but the resource requirements are pretty high for the "control server" component. https://www.nomadproject.io/docs/install/production/requirem... Nomad servers may ne…

I too was initially put off by those requirements. Now we run our Nomad server on a single t3.medium instance that sits at a 0.1 15 minute load average and has ~500MB of RAM used.

This manages about 100 client nodes. No need for a cluster since we don't need high availability on our control plane, and there's no actual state stored there that isn't created from our CI pipeline.

Re: Why Kubernetes is so complex

#30

Post by techdragon »

cb22 wrote: post #29
techdragon wrote: post #25

Earlier quoted context omitted.

I recently looked at Nomad, but ironically for a "less complex" piece of software, I couldn't be sure it would scale down to my use case, I ended up with K3S as it would run on the small hardware nodes i needed it to run on. I would have preferred Nomad but the resource requirements are pretty high for the "control server" component. https://www.nomadproject.io/docs/install/production/requirem... Nomad servers may ne…

I too was initially put off by those requirements. Now we run our Nomad server on a single t3.medium instance that sits at a 0.1 15 minute load average and has ~500MB of RAM used. This manages about 100 client nodes. No need for a cluster since we don't need high availability on our control plane, and there's no actual state stored there that isn't created from our CI pipeline.

See this is the sort of information that they should have posted on the requirements page. Not some arbitrary sized high water mark needing gigs of disk and memory. I'll have to give using Nomad (and Consul) another go next time, and test it on physical hardware to see.

Thanks for the info! and for doing what Hashicorp seemed too busy to do themselves.

Post reply on HN