Live data from Hacker News

Why is Kubernetes getting so popular?

stackoverflow.blog

661–670 of 681 posts

Re: Why is Kubernetes getting so popular?

#661
post #311

Earlier quoted context omitted.

Of course everyone's experience is different, but in my case... > How do you view all the VMs in a project across the globe at the same time? I'm not sure what it's got to do with k8s? I can't see jobs that belong to different k8s clusters at the same time, either. > Do you need to manage keys when ssh'n into a VM? Well, in k8s everybody who has access to the cluster can "ssh" into each pod as root and do whatever th…

If you're running pods as root, you're doing it wrong. That was a no-no with docker, and it's still a no-no for kubernetes. People still run non-containerized services as root too...

Docker won't run on my local machine as non root. And all the stuff I have seen so far has been run as root.

Re: Why is Kubernetes getting so popular?

#662

What does the Kubernetes configuration format offer over configuration management systems like Ansible, Salt, Puppet, Chef, etc?

Containers are the big difference. Kubernetes is one way to deploy containers. Configuration systems like Ansible/Salt/Puppet/Chef/etc are another way to deploy containers. Kubernetes also makes it possible to dynamically scale your workload. But so does Auto Scaling Groups (AWS terminology) and GCP/Azure equivalents. The reality is that 99% of users don't actually need Kubernetes. It introduces a huge amount of comp…

> It introduces a huge amount of complexity, overhead, and instability

I've never used it but might find myself using it in the future.. can you elaborate on these a bit? I'm curious what the pitfalls might be

Re: Why is Kubernetes getting so popular?

#663
post #61

Earlier quoted context omitted.

Then you need to add management of storage for it, management of logs, integration of monitoring, healthchecks, maybe some multiple environment case because UAT is good thing to have, etc. etc.

For most things I'd reckon you could skip most of that. Everyone thinks they need a Ferrari when a bike can then to work easily.

That's more "basic farm vehicle / lorry" than Ferrari.

You always have those concerns, it's just implemented differently. Customer hurling abuse at you over the phone (or worse - in person) is a form of healthchecks and monitoring, if worse than often common "have someone log in to the server every day and check if it's alive".

So is frantically logging into server to manually truncate log files that filled your one and only disk volume and caused the above abuse.

So is "we're losing customers because of how slow it is" yet not having a single idea why it is slow, because it runs fast when dev checks on their laptop.

All of the above are based on actual real world events, sometimes involving large corporations. In fact, the large corps seem to have most issues with manual work, because they can afford throwing cannon fodder ^W^W "experienced engineers" at the problems.

At some point it becomes a question of what is good use of your time. I disagree heavily with people claiming that running kubernetes is somehow orders of magnitude more complex than anything else (especially with k3s and using non-etcd backing stores). The complexity is necessary complexity, which you can tackle in various ways including YOLO.

Sometimes the YOLO approach however bites in the worst moment, and spending time on bespoke scripts, or figuring out configuration drift, are all costs that show up as you tackle said complexity.

Personally, the reason I went with kubernetes in the first production deployment I did with it, after being vocal anti-docker person at work, was because of... cost efficiency. Both in terms of my time (even though we had to spend significant amount of time migrating, as it was lift&shift of existing software), and in terms of compute costs - thanks to heavily loaded nodes our worst compute bill never reached above 20% of previous "condition normal". I don't think we ever really had more than 10 servers on purpose. Using k8s paid for itself.

Re: Why is Kubernetes getting so popular?

#664
post #639
post #538

Earlier quoted context omitted.

I'm logging into a server because i need to, not because its 'pleasure'. I don't hate it but if you need to login to a server regularly because you need to do an apt upgrade, you should have enabled automatic security updates and not login every few days. If your server runs full because of some logfiles or stuff, you should fix the underlying issue and not needing to login to a server. You should trust your machines…

Devs went on holiday before k8's.

Yes true, not sure what point you are trying to make.

Re: Why is Kubernetes getting so popular?

#665

Earlier quoted context omitted.

Even if one does have a single binary and config file that one can just copy to a server and run, there's more to non-trivial deployments than that. For example, how do you do a zero-downtime deployment where you copy over a new binary, start it up, switch new requests over to the new version, but let the old one keep running until either it finishes handling all requests that it already received or a timeout is reac…

Most web applications don't need any of that. Also, I didn't say k8s was useless, just that it's the new thing everyone wants (that they probably don't need).

I disagree about what most web applications need. It's not the 90s anymore. Everyone expects zero downtime and frequent updates.

Re: Why is Kubernetes getting so popular?

#666

Earlier quoted context omitted.

I got a bit disillusioned with k8s and looked at Nomad as an alternative. As a relatively noob sysadmin, I liked it a lot. Easy to deploy and easy to maintain. We've got a lot of mixed rented hardware + cloud VPS, and having one layer to unify them all seemed great. Unfortunately I had a hard convincing the org to give it a serious shot. At the crux of it, it wasn't clear what 'production ready' Nomad should look lik…

Nomad Team Lead here. Edit: just noticed an actual Nomad user replied as well, and I like their answer better. Consider mine an addendum. :) Batch workloads rarely require Consul, but for deploying your standard network services on Nomad: Consul is basically required. You could likely use any number of service mesh systems instead (either as sidecars, Docker network plugins, or soon CNI), but you'll be doing a lot of…

Thanks for reaching out! Since I have the chance I'll add - Nomad is pretty awesome, and I love the work your team is doing.

My org looked at Nomad at a time when there was a lot of pressure from above to deliver something as soon as possible. Two weeks just weren't enough to full lay of the land ¯\_(ツ)_/¯

Funny thing is even if I could plug in my own service discovery into Nomad, I would probably chuck it away and replace it with Consul after a few weeks anyway haha

Re: Why is Kubernetes getting so popular?

#667

Earlier quoted context omitted.

I got a bit disillusioned with k8s and looked at Nomad as an alternative. As a relatively noob sysadmin, I liked it a lot. Easy to deploy and easy to maintain. We've got a lot of mixed rented hardware + cloud VPS, and having one layer to unify them all seemed great. Unfortunately I had a hard convincing the org to give it a serious shot. At the crux of it, it wasn't clear what 'production ready' Nomad should look lik…

Nomad + Consul is the happy path. Adding Vault into the mix is nice, but not required. Consul by itself is the game-changer. Even in k8s it's a game-changer. It solves so many questions in an elegant way. "How do I find and reach the things running in (orchestrator) with (unknown ip/random port) from (legacy)?" being the most important. You run 5 servers, and a relatively lightweight client on everything (which isn't…

Interesting, thanks for sharing!

I'm assuming your team is using vault for PKI, but is there a similarly happy path for issuing certs without Vault.

I started off just using `openssl` but it all felt very janky, and I didn't really have any idea how CRLs should be setup

Re: Why is Kubernetes getting so popular?

#668

Earlier quoted context omitted.

Nomad + Consul is the happy path. Adding Vault into the mix is nice, but not required. Consul by itself is the game-changer. Even in k8s it's a game-changer. It solves so many questions in an elegant way. "How do I find and reach the things running in (orchestrator) with (unknown ip/random port) from (legacy)?" being the most important. You run 5 servers, and a relatively lightweight client on everything (which isn't…

Interesting, thanks for sharing! I'm assuming your team is using vault for PKI, but is there a similarly happy path for issuing certs without Vault. I started off just using `openssl` but it all felt very janky, and I didn't really have any idea how CRLs should be setup

Vault is great for just a PKI, even if you aren't using it for anything else. There are some tools that just do PKI, but Vault works a real treat at it. Any Terraform backend that supports encryption + Terraform + Vault gives you such an amazing workflow. We use a mix of short and long certs, with different roles based on what's getting a cert.

For now, we have CRLs disabled on all short-lived backends, enabled on long-lived backends and we're actually looking at disabling storing short-lived certs in the storage system at all, and just cranking the TTL down to really truly short. We've tested it as low as 30m, but a more real-world max-ttl is 1 week, with individual apps setting it as low as they can handle. For reference we run more than 10 PKI backends, and adding one (or a bunch) more is just a little terraform snippet for us.

The way it works via hashicorp template land, is that you just plop

    {{ with secret "name-of-pki/issue/name-of-role" "common_name=my.allowed.fqdn" "ttl=24h" }} {{ .Data.certificate }} {{ end }}
into your Nomad template stanza, or use consul-template directly as a binary, or use vault agent with it's template capability. You can get the CA chain if required the same way, just hitting a different PKI endpoint.

Also, as of Vault 1.4, Vault's internal raft backend is now production ready, making it a snap to run.

Try running through a few of the Vault quick-start guides, and replicating them in Terraform as much as possible. There's a few things TF does not handle gracefully last I checked (initial bootstrap), but you can get around that by using a null_resource or just handling that outside Terraform.

Re: Why is Kubernetes getting so popular?

#669
post #651

Earlier quoted context omitted.

Yaml is on the way out, Cuelang will replace it where it's used for infra. It's quite easy to start by validating yaml and then you quickly realize how awesome having your config in a well thought-out language is!

I thought you were trolling with some called Cuelang but is actually a thing. Yaml will still be used in 100 years, k8s is yaml based...

Yes but YOU won't be writing or seeing yaml, Cue will output the yaml and run kubectl (I'm already doing this to great relief)

Which is step 2 of my Enterprise adoption strategy. Step 1 is starting with validation, step 3 and beyond is where the real fun starts!

Re: Why is Kubernetes getting so popular?

#670

Earlier quoted context omitted.

Sure, the most asinine thing (borrowing from Rob Pike) is to have a system where invisible characters define the scope and semantics of what you are writing. Now Helm takes this one step further (and I one beyond that before saying no more to myself and discovering https://cuelang.org ) and starts using text interpolation with helpers for managing indentation in this invisibly scoped language. I hacked in imports, bu…

Just wanted to clarify that Dark isn't like Pulumi. The idea with Dark is that there is no configuration, just code.

There is still configuration, there has to be, you've just wrapped it so much it's not visible anymore (which is even worse than Pulumi, at least they are using an existing language). You still have to express (and write) the same information...

Darklang is solidly in the Pulumi camp, that's where outsiders put it. (I have seen the insides without beta / your demo, someone with a beta account showed me around a bit)

The real problem with Darklang is they have their own custom language and IDE. What exactly are you trying to solve?

Post reply on HN