Live data from Hacker News

The sad state of sysadmin in the age of containers (2015)

vitavonni.de

291–300 of 324 posts

Re: The sad state of sysadmin in the age of containers (2015)

#291
post #77

You've got to look at this in a context where platform package managers like apt are simultaneously 1) platform-specific 2) jealous, insisting that every language has to conform to their way of doing things and 3) fundamentally not very good, having very limited ability to do things like install packages for a single user or install multiple versions of the same package. Platform package managers like maven have been…

In my opinion there’s simply too many dependencies to begin with. Some of the simplest things put into a GitHub repository to be “shared” with the world is great—as a gist file. Software needs to be a little more self-contained. Software reuse these days is honestly taking “not reinventing the wheel” to lazy extremes. System package management systems do things just fine. Ask yourself, “when did I last properly packa…

> Software needs to be a little more self-contained. Software reuse these days is honestly taking “not reinventing the wheel” to lazy extremes.

The three cardinal virtues of a programmer are laziness, impatience and hubris. I've never seen a decent argument against using ever-smaller libraries (as long as our dependency management is good enough to track them); all of the usual arguments for reusing libraries still apply, even if we take them further than their originators imagined.

> System package management systems do things just fine. Ask yourself, “when did I last properly package a Debian library? Do I even know how to do it properly?” Most people won’t be able to answer that because they didn’t.

Well, as we're seeing, Debian is struggling to keep up with packaging of everything that users want. I have actually created some (unofficial) debian packages; it's fine, but it's not a particularly inspiring experience either. Creating a maven/pip/... package is generally a nicer experience.

> But truth be told, having each programming language a unique package manager sucks. I’d rather have a consistent way of managing packages on a given system so that I can use various programming languages.

In principle I agree. But I'd definitely need to be able to install independent copies of the same library, different packages for different users, and fundamentally just have a nice development experience when working on that package manager. And that's not something I see the sysadmin tradition being able to come up with.

Re: The sad state of sysadmin in the age of containers (2015)

#292
post #109

Earlier quoted context omitted.

> unwilling to play nice with "external" dependencies But that's a policy decision - the dependencies should themselves be packages, because that's the only way to guarantee reliability. The inability to install multiple versions is probably the only serious problem with the dpkg model. Even then we might argue that the dependent software should be fixed so that it's not so fussy about specific dependency versions.

> The inability to install multiple versions is probably the only serious problem with the dpkg model. Also can't install to separate disks.

That's unlikely. Worst case: examine where the package will put its files, and prep your system with symbolic links to directories residing on a file system on the other disk(s).

But that raises the question: why are your file systems sensitive to disk drives? Use a layer (or more) of indirection through LVM and RAID so to make the drive placement opaque and robust.

Re: The sad state of sysadmin in the age of containers (2015)

#293

Earlier quoted context omitted.

You have make call the platform specific build systems.

Wouldn't it just be like a build.sh? And makefile is not cross platform enough, especially on Windows.

I can't speak to the others, but if you've installed rubygems on Windows, you've installed mingW, and you have access to those tools. This is because rubygems will not build source gems against microsofts toolchain, but against the gcc toolchain ala mingW.

If you prefer using Powershell on Windows, great (or bat if you're a masochist). The point was that being in Ruby doesn't automatically invalidate the use of make as a build tool.

Re: The sad state of sysadmin in the age of containers (2015)

#294
post #286

Earlier quoted context omitted.

> jealous, insisting that every language has to conform to their way of doing things Let's ignore for a moment all your baseless assertions and focus on the following question: how do you ensure that a build is reproducible if you have no control on which required dependencies are downloaded by a build system which requires root access? Is jealousy the only explanation that comes up to you?

Reproducible builds were a non-goal for apt for most of its existence.

That doesn't dismiss its immense benefits. Besides, what's the source of your claim? Package quality, ease of install, and security were always high-priority topics for Debian AFAICT.

Re: The sad state of sysadmin in the age of containers (2015)

#295
post #263

Earlier quoted context omitted.

It feels like you're being unnecessarily hostile, here. He didn't address your specific problem as stated, but he did add to the discussion. I don't think it's fair to say that Nix is 'just another package manager'. It solves many of the stated problems with distro package managers (overlapping versions, user-specific packages, strict build environment rules), and provides many of the benefits of Docker and it's ilk…

> It feels like you're being unnecessarily hostile, here. He didn't address your specific problem as stated, but he did add to the discussion. You're right, sorry for the tone. The thing is, it sounded an awful lot like a blatant attempt at derailing the discussion by shoehorning span to promote a build tool. Nix does not solve anything, particularly as it was being proposed as a solution to a problem that plain old…

I like Apt a lot, but it does have some shortcomings.

First, of course, it requires an apt-based distro, so software distributors need to have apt alongside all the other packaging alternatives. Or they just provide a bash script. Ew.

Second, apt doesn't elegantly handle different versions of the same package. That's rarely an issue for well-established C libraries, but it's a big issue for Java and most of the dynamic languages. So you end up with a host of language-specific package managers.

Third, there's stuff beyond simple files that falls outside the wheelhouse of apt. Networking, configuration, whatever. The paradigm for apt is very much to have a small number of systems, manually curated by dedicated sysadmins. When you start scaling up to tens, hundreds, and thousands of hosts, you end up writing and maintaining long scripts to initialize a freshly-installed system and put it in the right state. And those script will break as packages evolve. Getting a system into a known-working state is difficult.

A specific example: I installed and set up GitLab on a Debian system a while back, and it was a huge pain. It's not just a package, after all, it's web code, a couple daemons, a sql db, a redis db, git repos on the filesystem, and more. The install guide was pages and pages long. I never quite got it working right (something about SSL certs, IIRC one of the daemons wasn't using the system CAs?).

So I tried docker for the first time, and had GitLab up and running in about 10 minutes.

And if I ever wanted to migrate to a different host, spin up another node for load balancing, or do backups and restores, you bet your ass I'd use docker.

Apt is great for carefully curated, individual systems. It was perfect for the world circa, say, 2005, and the world would be better of if we'd all standardized on it then. But even if we had, somebody would've invented something like docker in the meantime, for managing complex software (like GitLab) on tens, hundreds, or thousands of hosts.

But docker has all the issues pointed out in the article above, and more besides (every image is hundreds of megs, because it contains a full, running Linux system...that's just crazy).

Nix can do the package management thing that apt does so well, and it can also do the reproduceable, holistic system build thing that Docker does. It can also make management of language dependencies (i.e. Java JARs) much more clean and elegant.

It's seriously worth checking out.

Re: The sad state of sysadmin in the age of containers (2015)

#296
post #63

Earlier quoted context omitted.

"But we need to figure out the tooling and ecosystem story to build, verify and update container deployments securely." Already done: Solaris zones. Available in the SmartOS distribution near you. Combine with OS packaging, imgadm and vmadm commands for maximum impact.

I've spend 1 minute skimming some of the docs, but it's unclear whether these tools can do a better job than the docker-like tooling. The vmadm "build an image from scratch with a json file" doesn't help in building Hadoop any more than the oldest versions of docker did using a minimal image and a Dockerfile. How does provide visibility into every dependency inside an image? Especially when using 3rd-party-maintained…

It doesn't. He's a troll that claims solaris is the solution to every problem that he doesn't understand.

Re: The sad state of sysadmin in the age of containers (2015)

#297
post #284
post #244

Earlier quoted context omitted.

> I've worked with developers that have used git for 10 years who didn't fully realize what all of the 'git reset' options entailed While knowing some of them, if you use git reset on a frequent basis is expected, knowing where to find information on the other options is essential. I frequently go back and read through the man pages for various git commands so that I understand what will happen if I use a particular…

I give this to my devs: https://www.atlassian.com/git/tutorials/atlassian-git-cheats...

I believe it's more beneficial for developers to learn the underlying model of how git works (blobs, trees, commits, tags, etc) and how various commands deal with those underlying concepts. The Pro Git book git internals chapter is a very good place to start. That, in combination with the man pages for the commands they go over, will greatly enhance one's understanding of how git works.

Relying on cheat sheets to learn how to use git is not much different compared to learning how to use a programming language via Stack Overflow. In other words, you'll never develop a more thorough understanding of how the tool works and how to use it effectively.

Re: The sad state of sysadmin in the age of containers (2015)

#298

Earlier quoted context omitted.

Wouldn't it just be like a build.sh? And makefile is not cross platform enough, especially on Windows.

I can't speak to the others, but if you've installed rubygems on Windows, you've installed mingW, and you have access to those tools. This is because rubygems will not build source gems against microsofts toolchain, but against the gcc toolchain ala mingW. If you prefer using Powershell on Windows, great (or bat if you're a masochist). The point was that being in Ruby doesn't automatically invalidate the use of make…

Yeah, maybe it is true for Ruby development on Windows, but not for others, says Node.js. Developing Node.js on Windows is (comparatively) easy to do so, usually you only need to install `node` and `npm`, then optionally install `yarn` and `windows-build-tools`. None of these steps will give you GNU toolchain, so it won't available until you install it.

That is ehy I prefer OS-agnostic tools such as npm or npm+gulp for more complex build tools.

Re: The sad state of sysadmin in the age of containers (2015)

#299
post #95

Earlier quoted context omitted.

It can be done using package managers too, for example using Nix. The problem here is not related to distribution method but rather dependency hell in javascriptland.

I would say the main issue is that Nix is super difficult and requires you to intimately know your entire dependency tree down into Linux particularities. JavaScript makes matters worse, but even packaging a nontrivial Python package pulled directly from Pypi is often difficult. I want to like Nix, but it is far too pedantic to be practical. And this isn’t even mentioning the usability issues.

I think you only need to specify your immediate dependencies in nix?

Re: The sad state of sysadmin in the age of containers (2015)

#300
post #94

Earlier quoted context omitted.

It's not just a question of being in the official repos. Even if you're running a private repository, the deb tooling is not suitable for building many kinds of software: it's unwilling to play nice with "external" dependencies, and still single-platform and limited in its functionality.

> it's unwilling to play nice with "external" dependencies This baseless assertion is simply out right wrong. See for example Debian's docs on Private Package Archive: https://wiki.debian.org/DebianRepository/Setup

Hear hear. If anything .debs dont care enough where dependencies come from. As long as the package name matches you are good. I have no complaints about debian packaging. I've used lots of different ones. All official packages are signed and its easy to sign your own packages.

Maven, on the other hand, is plain scary.

Post reply on HN