81–90 of 102 posts
Setting aside the k8s content itself, I love the way this article is written. It's not a typical tutorial or tips/tricks but takes you time-traveling through the experience of a big company adopting nascent tech. Lot of great things to take away even outside of the kubernetes tips.
Earlier quoted context omitted.
> Mesos is inevitably fading as a legacy platform. Because of Chronos? This is a bizarre thing to say. Mesos actually works extremely well. Whenever I ask the why kube over Mesos question, I never get a good answer. I think because people just don’t know Mesos. Also it wasn’t made by google.
Chronos is just an example. There're many bugs in Mesos that don't get fixed for months/years. Mesos core is legacy (pre 11) C++ code nobody wants to maintain.
Disclaimer: Apache Mesos committer/PMC
Earlier quoted context omitted.
> Mesos is inevitably fading as a legacy platform. Because of Chronos? This is a bizarre thing to say. Mesos actually works extremely well. Whenever I ask the why kube over Mesos question, I never get a good answer. I think because people just don’t know Mesos. Also it wasn’t made by google.
Chronos is just an example. There're many bugs in Mesos that don't get fixed for months/years. Mesos core is legacy (pre 11) C++ code nobody wants to maintain.
I always search for mentions of Hashicorp Nomad in the comments section of front-page Kubernetes articles like this. There are often few or no mentions, so I’d like to add a plug for the Hashistack. For some reason Nomad seems to get noticeably less publicity than some of the other Hashicorp offerings like Consul, Vault, and Terraform. In my opinion Nomad is right up there with them. The documentation is excellent. I…
I have a few production Mesos clusters under my belt and one production Nomad and I really like Nomad and Mesos is not bad. Kubernetes seems to be a lot of magic and NIH and tries to do everything itself, whereas Mesos and Nomad are nicely composable and easy to reason about. Nomad's biggest benefit for me is a very nice integration with Vault (and Consul), I can have Nomad ask for a container instance specific secre…
Not heard that criticism before, what are you referring to in particular? The NIH part seems incongruous to me, since Google were a major contributor in inventing warehouse scale computing and cluster schedulers (c.f. the Borg and Omega papers, etc.).
Earlier quoted context omitted.
I would argue the biggest strength is maintainability. Managing and keeping up a distributed cluster with k8s is WORK. If you are not at the scale where you can dedicate full-time staff to managing only k8s, you shouldn't even be touching k8s. You need full-time staff to keep it alive. Nomad is operationally simple, you can run it out of your normal devops roles, you don't need dedicated staff. Mostly because you can…
I don't see why - I have my GCE cluster running fine with zero maintenance work.
I hope whatever you are running under k8s isn't crucial or important, and I really hope I'm not a customer of whatever you "operate".
Maintenance is real, that applies to everything if you want it to work reliably for any length of time. There are various ways to handle maintenance, do a little consistently and constantly (what most of us professionals do) or do large bulk-replacements every X time (like when stuff crashes and burns - and nobody can remember how to fix it, so they just replace it with whatever is new and shiny).
Earlier quoted context omitted.
I disagree that Jenkins is king for this. Jenkins is a single point of failure, is isn't a highly available distributed scheduler. It is a single master with slaves. While it is easy to configure Jenkins jobs with code (Job Builder, Job DSL, Jenkinsfiles), it is a pain to manage Jenkins itself with code. Plugins, authentication, all the non-job configuration, that is usually done via the GUI. Saying Jenkins can be co…
> it is a pain to manage Jenkins itself with code. Plugins, authentication, all the non-job configuration, that is usually done via the GUI. This is not true, all the configuration is scriptable via groovy scripts. We run bunch of groovy startup scripts that configure everything post launch. There is an effort to support this better[1] by jenkins team. > How to you isolate job executions in Jenkins? In Kubernetes eac…
> This is not true, all the configuration is scriptable via groovy scripts. [...] There is an effort to support this better[1] by jenkins team
The link you gave confirms it by saying managing Jenkins code "require you know Jenkins internals, and are confident in writing groovy scripts". Neither GUI's (like the one shown in your link) nor procedural languages (like Apache Groovy, still procedural even though its collection API is crippled for Jenkins pipelines) are very good for configuring software. Nor is an unreadable declarative language (like XML).
A readable declarative language (like YAML, as shown in your link) is the solution. Languages like Groovy were an over-reaction against unreadable XML in the Java ecosystem. The correct solution is to switch from an unreadable to a readable declarative language for configuring software.
Earlier quoted context omitted.
Setting up a K8S cluster isn't that hard actually. From my experience, the hard part kickin when dealing with stateful service which needs to associated with volume. Even with a managed cluster, you still have to solve that problem. Either you pre-provision disk or use dynamic volume. Next is when upgrading K8S version. with a stateless service, it's a walk in a prt to upgrade. With data volume it's more tricky to up…
That's true, but I'm not sure if using Jenkins would avoid these problems you outline. And that's really the crux of what the OP is suggesting; that Jenkins or something smaller than Kube would have been a better choice.
Other than that I agree with you.
Earlier quoted context omitted.
> it is a pain to manage Jenkins itself with code. Plugins, authentication, all the non-job configuration, that is usually done via the GUI. This is not true, all the configuration is scriptable via groovy scripts. We run bunch of groovy startup scripts that configure everything post launch. There is an effort to support this better[1] by jenkins team. > How to you isolate job executions in Jenkins? In Kubernetes eac…
> > it is a pain to manage Jenkins itself with code > This is not true, all the configuration is scriptable via groovy scripts. [...] There is an effort to support this better[1] by jenkins team The link you gave confirms it by saying managing Jenkins code "require you know Jenkins internals, and are confident in writing groovy scripts". Neither GUI's (like the one shown in your link) nor procedural languages (like A…
I somewhat agree with you. Unfortunately Jenkins team seems to have bet in the opposite direction by going full groovy https://github.com/jenkinsci/pipeline-examples
Earlier quoted context omitted.
I don't see why - I have my GCE cluster running fine with zero maintenance work.
Zero maintenance work implies you are not doing security patches or upgrades, so as soon as you have a problem, not only will you be left holding the now broken pieces, nobody will have any reason to help or support you, unless you pay them $$$$$$'s(and even then.... maybe not). I hope whatever you are running under k8s isn't crucial or important, and I really hope I'm not a customer of whatever you "operate". Mainte…
Much as it burns me to admit this, for this usecase, jenkins is king. At previous job, we had migrated from a nasty cron orchestration system to jenkins. It did a number of things including building software, batch generating thumbnails and moving data about on around 30 nodes, of which about 25 were fungible. Jenkins job builder meant that everything was defined in yaml, stored in git and was repeatable. A sane user…