231–240 of 446 posts
Earlier quoted context omitted.
Docker is heavier (and more dangerous) because of dockerd, the management and api daemon that runs as root. Actual process isolation is handled by cgroup controls which are already built into the kernel and have been for years. You can apply them to any process, not just docker ones. However, Docker is essentially dead; the future is CRI-O or something similar which has no daemon and runs as an unprivileged user. And…
All the so-called "docker killers" are essentially unfinished products. They don't compare 1:1 to docker in feature set and even if they run as rootless, they still are vulnerable to namespace exploits in the Linux kernel. Though docker runs as root, it's still well protected out-of-the-box for the average user and is a very mature technology.
Earlier quoted context omitted.
All the so-called "docker killers" are essentially unfinished products. They don't compare 1:1 to docker in feature set and even if they run as rootless, they still are vulnerable to namespace exploits in the Linux kernel. Though docker runs as root, it's still well protected out-of-the-box for the average user and is a very mature technology.
One of the biggest benefits of k8s for me, back in 2016 when I first used it in prod, was that it threw away all the extra features of Docker and implemented them directly by itself - better. Writing was already in the wall that docker will face stern competition that doesn't have all of its accidental complexity (rktnetes and hypernetes were a thing already)
Earlier quoted context omitted.
> What works for us is to do the simplest thing that works, then iterate. The older I get, the more often I'm reminded that this un-sexy approach is really the best way to go. When I was younger, I always thought the old guys pushing boring solutions just didn't want to learn new things. Now I'm starting to realize that after several decades of experience, they simply got burned enough times to learn a thing or two h…
Jonathan blow had a great take on this that really spoke to me. I can’t do it justice, but paraphrasing as best I can: Get the simple to understand, basic thing working and push the harder refactor / abstraction until later. Leave the fancy stuff for a future engineer who better understands the problem. That future engineer is you with more experience with the actual problem.
Like, there's no point in scratching away at your own web CMS just to prove to yourself that you really do need Rails or Django. Especially when part of the purpose of a framework is to guide your design toward patterns which will fit with the common, preexisting building blocks that you are likely to end up needing. If you do the design separately, you risk ending up in a place where what you've built isn't really suited to being adapted to other stuff later.
For a humble example of this from myself, I built an installer system for Linux computers— initially very simple, just a USB boot, perform partitioning, unpack a rootfs tarball, and set up the bootloader. Then we added the ability to do the install remotely using kexec. Then we added an A/B partition scheme so there would always be fallback. It got more and more complicated and fragile, and the market around us matured so that various commercial and OSS solutions existed that hadn't when we first embarked on this effort. But the bigger and less-maintainable the homegrown system became, the more features and capabilities it had which it made it difficult to imagine ditching it in favour of something off the shelf. There isn't really a moral to this story other than it would be even worse if we'd built the in-house thing without doing a thorough evaluation of what was indeed already out there at the time.
(small SaaS, max out at ~100k concurrent users right now, but growing fast)
Earlier quoted context omitted.
> That's the easy part! The hard part is managing databases. Ding ding ding ding ding. "But what about disk?" (so, relatedly, databases) is the hard part. Balancing performance (network disks suuuuuck) and flexibility ("just copy the disk image to another machine" is fine when it's a few GB—less useful when it's lots of GB and you might need to migrate it to another city and also you'd rather not have much downtime)…
Sometimes I think K8s is largely pushed for Amazon/Google/Microsoft to sell the disk that goes along with it.
Earlier quoted context omitted.
It's 'easy' to do it with rsync and Make as long as you don't have a lot of dependencies. Once you start pulling in dependencies, you have to figure out how to get them installed, hopefully before you roll code that uses them. If the dependencies are small, you can pull them into your source (more or less) and deploy them that way, but that may make tracking upstream harder. (otoh, a lot of things I personally used l…
Just curious, would your deployment method be any different today? If so, how?
Of course, today, my fleet is down to less than ten hosts, depending on exactly how you count, and I'm usually the only user, so I can do whatever.
Containers yes but nothing else. Running 1B valuation with manually going to instances and docker pull xxx && docker-compose down && docker-compose up -d. EC2 created by hand. No issues.
We use freebsd jails and a lightweight in house orchestration tool written in Rust. We are running hundreds of Ryzen machines with 64 cores. Our costs compared to running equivalent on Amazon is so much less. We estimate our costs are about 6x lower than AWS and we have far better performance in terms of networking, CPU, and disk write speed. Jails has been a pleasure to work with! We even dynamically scale up and do…
It was run on naked boxes in many pops worldwide. We heavily used the jvm, which dramatically simplifies application distribution. Boxes were all netboot; they came up, grabbed the OS, and based on some pretty simple config figured out what jars to download and run.
We also costed out a move to aws, and it was somewhere between 5 and 6x as expensive as buying our own hardware.
Thousands of boxes worldwide managed with a team of two fulltime sysadmins plus two more primarily engaged in dev with sysadmin duties as-needed, plus remote hands services in the various datacenters.
Earlier quoted context omitted.
Do your developers run the same Nix packages that you deploy to production? (that sounds like a energy-level-transition in developer productivity and debugging capability, if so)
Yeah, the environment is bit-for-bit identical in dev and prod. Any difference is an opportunity for bugs. OK, there's one concession, there's an env var that indicates if it's a dev and prod environment. We try to use it sparingly. Useful for stuff like not reporting exceptions that originate in a dev environment. Basically, there's a default.nix file in the repo, and you run nix-shell and it builds and launches you…