91–100 of 163 posts
Earlier quoted context omitted.
Uhhh... backups?! Not all companies release daily, weekly, and sometimes not even monthly. Stage the rollout, do your testing, get your evidence, get your plan, perform the release.
So if you deploy a new release which turns out to be buggy, your only recourse is doing a full backup restore?
I'm still relatively new to Linux. My question is why wouldn't Fedora Server be considered an alternative for the CentOS diaspora? It seems a better fit than Debian to me.
Previous product I worked on required a supported RHEL/CentOS environment on which to install our product. Even with the six years or so of support, our customers would piss and moan every time we'd tell them that RHEL/Centos5 was hitting EOL and they'd need to upgrade their servers to Centos 6 or 7 to stay supported. Most of them wouldn't even entertain the idea of "upgrading": for them, business as usual was holdin…
Earlier quoted context omitted.
Re: Containers aren't quite there yet. I think they've been there for 12-14 months. It's no longer a question of "If" but "When" a company decides on its container strategy (and its more than just k8s - see https://blog.coinbase.com/container-technologies-at-coinbase... ) I work for a company that is 100% k8s. Base linux of the containers is Debian 8- but honestly doesn't really matter that much - the OS is more kube…
When I was working in a stack like this I found people spending outstanding amounts of time not actually working to improve the stability/performance of the application. The reason you triggered that memory was "GKE, Prometheus, Sysdig, Grafana, ELK" - that's exactly what we were dealing with. The support infrastructure/compute needs for it far exceeded the 20-30 hosts that actually needed to be there to operate the…
K8S is in flux, so you still have to have a few top-end SRE types to manage your kube environment - the acceleration / maturity of the ecosystem is incredible though, so, sometime in the next 3-4 years, we'll start to see things get standardized enough that the wizardry required to keep it running will become a more commodity skill set.
And, more importantly, most of the ecosystem is fairly identical between azure/google/AWS - so porting or going multi-cloud is usually a weeks effort if that's something you want to do.
By "Moving up the Stack" - Of course I understand that cgroups/linux underpins it all - it's just that we're not using linux system binaries to manage the containers directly.
I mean tasks like process, storage, memory, CPU, resource utilization isn't something we tweak/query with OS commands, rather we're sending request/limit configurations to kube, and let it worry about managing the resources, relying on PromQL to monitor resource utilization, etc...
Earlier quoted context omitted.
To be fair, containers often end up being its own Linux installation with its own configuration drift. So many dockerfiles mindlessly pull in an entire Ubuntu system just to run a simple app.
But the image [1], once built, is still idempotent. You can deploy it and it will always contain the same configuration and code. Meanwhile, a month-long Ubuntu VM that has received regular CM pushes (including system updates) will likely vastly differ from a branch new Ubuntu VM and a single CM push. To the point, where you can't be sure anymore that your current CM config will even work on a brand new machine, unle…
Earlier quoted context omitted.
I'm really considering doing this for a new spin-up myself... The stability of Debian tooling for large-scale, hyper scaleable solutions is outstanding. I just feel like the world balks at me every time I do something considered slightly "old fashioned" completely ignoring the finished product to shame me for not using buzz-word tooling. Containerization is fantastic don't get me wrong, but I've had more success with…
Debian plus an automation system like Puppet, Chef, etc works extremely well. It's just not sexy enough for people to write hundreds of posts about how to set up your own package repo, understand unattended-upgrades, and do monitoring.
Eh, it does until it doesn't. Sooner or later you run into pitfalls around the leaky abstraction of pretending your state is truly idempotent and path-independent. E.g. spinning up a new instance works fine, but the existing instances that need to uninstall a previous version to upgrade to the newer one end up breaking. Or vice-versa, existing servers work fine but then when you need to launch a new one your realize the config no longer works on a clean install and you hadn't noticed it for weeks.
Container-based systems certainly have their own problems, but it is really nice having a model where you don't allow long-lived implicit state and cruft to accumulate on your application servers in the same way.
Earlier quoted context omitted.
I'm really considering doing this for a new spin-up myself... The stability of Debian tooling for large-scale, hyper scaleable solutions is outstanding. I just feel like the world balks at me every time I do something considered slightly "old fashioned" completely ignoring the finished product to shame me for not using buzz-word tooling. Containerization is fantastic don't get me wrong, but I've had more success with…
Debian plus an automation system like Puppet, Chef, etc works extremely well. It's just not sexy enough for people to write hundreds of posts about how to set up your own package repo, understand unattended-upgrades, and do monitoring.
Doing this is hard and there’s nothing in the ecosystem I’m aware of to guarantee that. Nix maybe? That’s part of the problem. The other is that docker has a conveniently large set of configs and things “out of the box” so there’s familiarity and documentation.
It’s not impossible to accomplish but building that community of doing things a sustainable way that actually addresses the pain points (rather than just dismissing it with “you’re just using the old tools incorrectly”) is what you need. If you’re going after docker your solution will have to support devs and devops. If you’re going for a niche community of enthusiasts/experts (probably more defensible and easier to grow), then focus just on a single niche use-case that general solutions could never outmatch (but don’t ignore growing it carefully if your niche solution is meaningfully better - listen to your users that you trust).
I somehow missed that Red Hat started sponsoring CentOS a while back, and owns the trademarks. I mean, money and cooperation is great and all, but how could anyone have expected CentOS Linux to continue for long when Red Hat has such a fundamental conflict of interest? Edit: here's the HN post at the time: https://news.ycombinator.com/item?id=7019914
IIRC the main maintainer had started working for RedHat (and didn't disclose that) before CentOS became a Red Hat community project.
> With containers, developers can develop, test, deploy, and rollback with blazing fast velocity. Wow, that has not been my experience with containers.
It's fast enough, that some (obv not most) developers don't even set up their local docker environment, and just use the CI/CD deployments to test their work. With the advantage being that if it looks good - single deploy moves all of the production environment to it.
Earlier quoted context omitted.
Yeah, I assume this ends with somebody forking CentOS into another "redhat without redhat branding and license costs".
Maybe CERN will revitalize Scientific Linux.
Earlier quoted context omitted.
When I was working in a stack like this I found people spending outstanding amounts of time not actually working to improve the stability/performance of the application. The reason you triggered that memory was "GKE, Prometheus, Sysdig, Grafana, ELK" - that's exactly what we were dealing with. The support infrastructure/compute needs for it far exceeded the 20-30 hosts that actually needed to be there to operate the…
I mostly agree with all of what you've said here. In our case, it's not unusual for a single customer environment to surge to 200-300 instances of an underlying compute server, and then scale back down to 20-30 at steady state. With 30 customer environments, you might have customers running from anywhere as low as 15 containers to as many as 500+, with a lot of dynamic flux depending on data ingestion and ETL. K8S is…
I am so ready for this!
Constant scale-up/scale-down and dynamic load is what I jump at K8s for personally. Totally see the use-case for what you're talking about.
All-in-all I love K8s and containers, use them myself, and have been really happy with the results. It's just when I've worked with it professionally I don't find my colleagues typically have the skillset (not the fault of the tech).