Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Convergence to Kubernetes

medium.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Convergence to Kubernetes

#71

Post by fnord123 »

pageald wrote: post #47
hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

Thoughts on Nomad after admittedly minimal dabbling: I pushed for using Nomad at my job without success. Managerial perceptions of Kubernetes as "the consensus" is a self-fulfilling prophecy. Nobody wants to pick a technology with a fraction of the buy-in of Kubernetes or be forced into paying boku bucks for an enterprise contract. Hashicorp's insistence on releasing premium closed-source features doesn't work when i…

>be forced into paying boku bucks

Boku is Japanese for "I" (cf Watashi).

Beaucoup is French for "a lot".

Re: Convergence to Kubernetes

#72

Post by KaiserPro »

ordinaryperson wrote: post #2

Dumb question from someone who doesn't use Kubernetes (or Docker) in production: don't routine security updates mean you're constantly rebuilding and redeploying these images? And if so, how is that more efficient than just using Puppet / Chef / Ansible and a 'real' server?

Yes, but its less of an issue mainly because there is an orchestration layer over the top of the deployment layer.

Basically there is a thing that spins up machines, connects them to a thing that places programs on them to run. The scheduler then starts your program and maps in storage, outside world connections (HTTP reverse proxy, or a bit more advance and some level 7 routing [ie /v1 goes to progam a /v2 goes to b])

when it works it means that the developer really doesn't need to think tomuch about _where_ a program runs.

Basically its a feature incomplete mainframe clone.

Re: Convergence to Kubernetes

#73

Post by icebraining »

lkrubner wrote: post #23
ordinaryperson wrote: post #2

Dumb question from someone who doesn't use Kubernetes (or Docker) in production: don't routine security updates mean you're constantly rebuilding and redeploying these images? And if so, how is that more efficient than just using Puppet / Chef / Ansible and a 'real' server?

Docker is a dangerous gamble and you can get more of an automated build system, with less devops effort, from Terraform and Packer. Avoid containers and stick with real servers “baked” by Packer: http://www.smashcompany.com/technology/docker-is-a-dangerous...

Docker aside, your arguments regarding fat binaries are kinda off mark. Python has been able to package an app and all of its dependencies into a single zip file - which is what a jar file really is - since before Go and Clojure even appeared (see PEP 273). Single-file packaging in JS is also common - very much used for browser deployment.

But "let's change languages because we don't have a reliable way to copy more than one file" sounds a bit insane to me. My solution was Debian packages, not Docker, though.

Re: Convergence to Kubernetes

#74

Post by falsedan »

hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

I haven't used Kubernetes yet since I have a Mesos scheduler that runs heavy workloads on-demand and wants to use all unused resources for a few minutes. I can't see a way to configure a k8 service to use either zero or as many pods as it possibly could, without writing something to watch the cluster's metadata and update the service's config files

Re: Convergence to Kubernetes

#75

Post by manigandham »

tannhaeuser wrote: post #45
manigandham wrote: post #22

Earlier quoted context omitted.

Scale isn't everything. Removing "hand-managed" from the entire process is an even bigger benefit. It's really not that complicated, it's clustering software that lets you encapsulate your applications into containers and just write simple declarative YAML files while it takes care of actually running it as specified, and keeping it running regardless of what happens to the hardware. Given all the managed offerings n…

> simple declarative YAML files If the config files and underlying infrastructure are changing all the time and have varying degrees of documentation, that's just shifting know-how from established tools to the newest fad, especially when k8s know-how is extremely scarce/expensive, and will leave you in a trial-and-error situation with unclear diagnostics if anything goes wrong due to the sheer complexity. Automation…

> k8s is just oversold and overhyped

Yes, like anything else is, but it also removes almost all ops overhead and is much faster and easier to maintain at the container level than recreating images and redeploying VMs.

Since the vast majority of startups just want to run apps and aren't doing any sophisticated lower-level infrastructure, trading terraform/chef/puppet for k8s yaml is a net win, especially since it also replaces several other accessory software like load balancers, service discovery, rolling deployments, etc. that you might need otherwise.

Re: Convergence to Kubernetes

#76

Post by dewey »

fnord123 wrote: post #71
pageald wrote: post #47

Earlier quoted context omitted.

Thoughts on Nomad after admittedly minimal dabbling: I pushed for using Nomad at my job without success. Managerial perceptions of Kubernetes as "the consensus" is a self-fulfilling prophecy. Nobody wants to pick a technology with a fraction of the buy-in of Kubernetes or be forced into paying boku bucks for an enterprise contract. Hashicorp's insistence on releasing premium closed-source features doesn't work when i…

>be forced into paying boku bucks Boku is Japanese for "I" (cf Watashi). Beaucoup is French for "a lot".

https://www.urbandictionary.com/define.php?term=Boo%20Koo%20...

Re: Convergence to Kubernetes

#77

Post by hardwaresofton »

falsedan wrote: post #74
hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

I haven't used Kubernetes yet since I have a Mesos scheduler that runs heavy workloads on-demand and wants to use all unused resources for a few minutes. I can't see a way to configure a k8 service to use either zero or as many pods as it possibly could, without writing something to watch the cluster's metadata and update the service's config files

Kubernetes has resource limiting built in, in the form of Resource Quotas[0] and more practically (or more straight forward to use) resources for pods[1]. The system's not perfect, so it's likely possible to overcommit machines, but there are QoS controls for critical workloads.

Kubernetes is built in a way that it's easy for people to write something that watches cluster metadata and perform actions (controller pattern), as such a lot of functionality is built in gradually overtime just like that. I'm not sure when resource management first came on the scene but it's been around for a while.

You can also use Kubernetes to manage resources completely unrelated to Kubernetes by bringing Custom Resource Definitions ("CRDs") into play -- that's when you create a "fake" Kubernetes resource that (ex. VirtualMachine) manages some resource that's actually on the machine. The combination of CRDs and controllers to manage them is called the "Operator pattern"[2] and it's gaining a lot of hype right now as people wrestle with it conceptually but it's been around the whole time.

[0]: https://kubernetes.io/docs/concepts/policy/resource-quotas/

[1]: https://kubernetes.io/docs/concepts/configuration/manage-com...

[2]: https://coreos.com/blog/introducing-operators.html

Re: Convergence to Kubernetes

#78

Post by jakoblorz »

minieggs wrote: post #40
hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

I would love to see a comparison between Docker Swarm and Kubernetes. From talking to peeps I've got that Kubernetes is better for a large number of nodes? I self host all my side projects with Docker Swarm and it's been so good I haven't needed to look into other container management solutions (but I've only got eight nodes).

I believe this does a good job comparing without going to deep, e.g. for people just getting started on swarm/k8

https://platform9.com/blog/kubernetes-docker-swarm-compared/

Edit: updated url to a more recent version of the article

Re: Convergence to Kubernetes

#79

Post by falsedan »

hardwaresofton wrote: post #77
falsedan wrote: post #74

Earlier quoted context omitted.

I haven't used Kubernetes yet since I have a Mesos scheduler that runs heavy workloads on-demand and wants to use all unused resources for a few minutes. I can't see a way to configure a k8 service to use either zero or as many pods as it possibly could, without writing something to watch the cluster's metadata and update the service's config files

Kubernetes has resource limiting built in, in the form of Resource Quotas[0] and more practically (or more straight forward to use) resources for pods[1]. The system's not perfect, so it's likely possible to overcommit machines, but there are QoS controls for critical workloads. Kubernetes is built in a way that it's easy for people to write something that watches cluster metadata and perform actions (controller patt…

Yeah with bare mesos I don't have to do any of that, I just accept resource offers and release them when I'm done. The scheduler decides how much it will scale, not mesos.

We like to create a new framework for each instance of the job (so we can track slow runs, trial-run new versions), so we can have anywhere between 0 and 50 similar frameworks all running and vying for 100% of the cluster.

Naturally, we have cluster-level scaling, so we add nodes until we hit our max spend per hour, and then jobs take a bit longer to complete.

Re: Convergence to Kubernetes

#80

Post by hardwaresofton »

falsedan wrote: post #79
hardwaresofton wrote: post #77

Earlier quoted context omitted.

Kubernetes has resource limiting built in, in the form of Resource Quotas[0] and more practically (or more straight forward to use) resources for pods[1]. The system's not perfect, so it's likely possible to overcommit machines, but there are QoS controls for critical workloads. Kubernetes is built in a way that it's easy for people to write something that watches cluster metadata and perform actions (controller patt…

Yeah with bare mesos I don't have to do any of that, I just accept resource offers and release them when I'm done. The scheduler decides how much it will scale, not mesos. We like to create a new framework for each instance of the job (so we can track slow runs, trial-run new versions), so we can have anywhere between 0 and 50 similar frameworks all running and vying for 100% of the cluster. Naturally, we have cluste…

It was kind of hard to understand what you were describing without reading the Mesos Architecture documentation[0], but I think I get it now. When you say "you" just accept resource offers, you mean the frameworks (in mesos terminology) you're deploying correct?

Weirdly enough, Mesos resembles a system I was building in my head that I thought could compete with Kubernetes... Taking the resource supplier (agents in mesos-speak) and consumer (frameworks in mesos-speak) paradigm to the extreme.

[0]: https://mesos.apache.org/documentation/latest/architecture/

Post reply on HN