71–80 of 156 posts
Earlier quoted context omitted.
Docker adds other value to the lifecycle of your deployment. An "optimization" where you're removing value is just a compromise. Otherwise we'd all run our static sites on UEFI.
What value is that, for running such a simple piece of software?
The ability to pull the image on to any machine without needing to clone the source files and build it.
Smaller images mean faster pod starts when you auto scale.
Earlier quoted context omitted.
Just because a project has a larger budget, doesn't mean any of it should be spent on Docker, Docker Swarm, or Kubernetes or whatever other managers of Docker that you can mention here. Fact is, for 3 servers, it would be hard to convince me of any use of Docker compared to the aforementioned deployment shell script + Debian unattended-upgrades. What problem does Kubernetes address here? So what if it is "easy to use…
And maybe if my grandmother had wheels she'd be a bicycle. But - over a so far 20-year career of which "devops" work has been sometimes a fulltime job, and always a significant fraction, since back when we called that a "sysadmin" - I've developed sufficiently intimate familiarity with both sorts of deployment workflows that, when I say I'll always choose Docker where feasible even for single-machine targets, that's…
Well, here is where I must thank you. Thank you for replying to me, and giving me a real reason to look at this ecosystem.
My personal deployment script is really just one SCP command - it copies the new version of my statically-built blog to my server. The web server comes with the OS, and that's all I need.
But when I read "hundred lines? A couple hundred?" I realized my company has a script fitting that bill. There might be an opportunity for improving it. While I am still somewhat skeptical, because using Kubernetes instead of that script might still not be worth it long-term for a 7-person company (of which 3 devs and one sysadmin), I will check out its capabilities.
> in so doing you may find yourself better able to inform your objections.
Thank you for the patience to follow up, in spite of my arrogance. I might just come up with an improvement somewhere. Certainly we're a long way from a billion in ARR - I am thankful for your valuable time, and wish you continued and further success!
Earlier quoted context omitted.
That's true, but in my experience there is nothing mutually exclusive in systems being simple and systems running Docker. Granted, you do need to learn how Docker works, and be ready to help others do likewise if you're onboarding folks with little or no prior experience of Docker to a team where Docker is used. That's certainly a tradeoff you face with Docker - just as with literally every other shared tool, platfor…
Docker is absolutely ups the complexity. Whatever set-up your application needs is a still necessary step in the process. But now you've not only added more software in docker with its a docker registry, and Docker's state on top of the application's state, you've also introduced multiple virtual filesystems and a layer of mapping between those and locations on the host, mappings between the container's ports and the…
It's rare that I need to find out de novo where a given path in a container is mapped on the host. When I do need to do that, I can usually check a deployment script, or failing that inspect the container directly and see what volume mounts it has.
I don't need to worry about finding paths very often - much less frequently than I need to think about deployments, which at absolute minimum is once per project.
So, sure, by using Docker I've introduced a little new complexity, that's true. But you overlook that this choice does not exist in a vacuum, and that that added complexity is more than offset by the reduction of complexity in tasks I face much more often than the one you describe.
And that's just me! These days I have a whole team of engineers on whose behalf, as a tech lead, I share responsibility for maintaining and improving developer experience. Do you think I'd do them more of a favor by demanding they all comprehend a hundred-line sui generis shell script for deployments, or by saying "here's a single command that works in exactly the same way everyone you'll work with in the next ten years does it, and if it breaks there's fifty people here who all know how to help you fix it"?
I don't know why there is a big fish at the top of your website, but I like it a lot.
Also the other blog posts have different big fishes, so check them out as well.
While this is remarkably a good hack and I did learn quite a bit after reading the post, I'm simply curious about the motivation behind it? A docker image even if it's a few MBs with Caddy/NGINX should ideally be just pulled once on the host and sit there cached. Assuming this is OP's personal server and there's not much churn, this image could be in the cache forever until the new tag is pushed/pulled. So, from a "h…
It gets pulled once per host, but with autoscaling hosts come and go pretty frequently. It's a really nice property to be able to scale quickly with load, and small images tend to help with this in a variety of ways (pulling but also instantiating the container). Most sites won't need to scale like this; however, because one or two hosts is almost always sufficient for all traffic the site will ever receive.
Even then, saving a few MBs in image size is the devops parlance of early optimisation.
There's so much that happens in an Autoscale group before the instance is marked healthy to serve traffic, that an image pull of few MBs in the grand scheme of things is hardly ever any issue to focus on.
Earlier quoted context omitted.
And maybe if my grandmother had wheels she'd be a bicycle. But - over a so far 20-year career of which "devops" work has been sometimes a fulltime job, and always a significant fraction, since back when we called that a "sysadmin" - I've developed sufficiently intimate familiarity with both sorts of deployment workflows that, when I say I'll always choose Docker where feasible even for single-machine targets, that's…
> what, a hundred lines? A couple hundred? You have to deal with different repos for different distros, I expect, adding repositories for deps that aren't in the distro repo, any number of weird edge cases Well, here is where I must thank you. Thank you for replying to me, and giving me a real reason to look at this ecosystem. My personal deployment script is really just one SCP command - it copies the new version of…
One thing, I'd suggest looking to Docker before Kubernetes unless you already know you need multi-node (ie, multi-machine) deployments, and maybe as a pilot project even if you do. Kubernetes builds upon many of the same concepts (and some of the same infrastructure) as Docker, so if you get to grips with Docker alone at first, you'll likely have a much easier and less frustrating time later on than if you come to Kubernetes cold. (And when that time does come, definitely start with k3s if you're doing your own ops/admin work - it's explicitly designed to work on a smaller scale than other k8s distributions, but it also pretty much works out of the box with little admin overhead. As with starting on Docker alone vs k8s, it's all about managing your frustration budget so you can focus on learning at a near-optimal rate.
But hey, thanks for the well-wishes, and likewise taking the time in this thread! It's been of real benefit to me as well. If we're to be wholly honest, as an IC in my own right I've never been above mid-second quartile at absolute best, and at my age I'll never be better than I am today - or was yesterday. But that also means I'm at a point in my career where I best spend my time helping other engineers develop; if I can master the skill of making my accumulated decades of experience and knowledge, and whatever little wisdom may be found in that, of use to engineers who still have the time and drive to make the most of it in ways I maybe failed to do - well, I'll take it, you know? It's not the kind of work I came into this field to do, I suppose, but I've done enough of it by now to know both that I can do it, and that it is very much worth doing.
So, in that spirit and quite seriously meant - I might be off work sick this afternoon, one peril I'm finding attends ever more frequently upon advancing age, but evidently that's no barrier to improving the core skill that I intend to build the rest of my career around. Thank you for taking the time and trouble to help make that possible today, and here's likewise hoping you find all the success you desire!
While this is remarkably a good hack and I did learn quite a bit after reading the post, I'm simply curious about the motivation behind it? A docker image even if it's a few MBs with Caddy/NGINX should ideally be just pulled once on the host and sit there cached. Assuming this is OP's personal server and there's not much churn, this image could be in the cache forever until the new tag is pushed/pulled. So, from a "h…
We could be using container snapshots/checkpoints so we don't need to go through as much initialization code. This would imply though that we configure via the file-system or something we can attach late though. Instead of 12-factor configure via env vars, as is standard/accepted convention these days. Actually I suppose environment variables are writable, but the webserver would need to be able to re-read it's config, accept a SIGHUP or whatever.
We could try to pin some specific snapshots into memory. Hopefully Linux will keep any frequently booted-off snapshot cached, but we could try & go further & try to make sure hosts have the snapshot image in memory at all times.
I want to think that common overlay systems like overlayfs or btrfs or whatever will do a good job of making sure, if everyone is asking for the same container, they're sharing some caches effectively. Validating & making sure would be great to see. To be honest I'm actually worried the need-for-speed attempt to snapshot/checkpoint a container & re-launch it might conflict somewhat- rather than creating a container fs from existing pieces & launching a process, mapped to that fs, i'm afraid the process snapshot might reencode the binary? Maybe? We'd keep getting to read from the snapshot I guess, which is good, but there'd be some duplication of the executable code across the container image and then again in the snapshotted process image.
Earlier quoted context omitted.
I’m not suggesting it has value to everyone. I’m suggesting it has value to the people who see value in it.
I'm super curious to know what the value to people who see that happens to be. It's serving static websites, why do I need to wrap THAT of all things in a container? Really, enlighten me
But once you start wanting to run a significant number of things, or a significant number of instances of a thing, it becomes more helpful to have a all-purpose tool designed to manage images & run instances of them. Having a common operational pattern for all your systems is a nice, overt, clean, common practice everyone can adapt & gain expertise in. Rather than each project or company defining it's own deployment/operationization/management patterns & implementations.
The cost of containers is also essentially near zero (alas somewhat less true with regards to local FS performance, but basically equal for many volume mounts). They come with great features like snapshots & the ability to make images off images- CoW style capabilities, the ability to mix together different volumes- there's some really great operational tools in containers too.
Some people just don't have real needs. For everyone else...
I don't know why there is a big fish at the top of your website, but I like it a lot.
https://nypl.getarchive.net/media/salmo-fario-the-brown-trou...