Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sad state of sysadmin in the age of containers (2015)

vitavonni.de

Post reply on HN
Print view View original

301–310 of 324 posts

Re: The sad state of sysadmin in the age of containers (2015)

#301

Post by james-mcelwain »

Russtopia wrote: post #261

Amen. I avoid any container-based or blindly 'curl |sudo' install (the latter only after manual inspection of such a script). Apps which only offer containers are untrustworthy black boxes. And it indicates devs who are too ignorant and/or lazy to make even basic efforts at distribution-neutral or portable code. Gentoo and Funtoo keep devs honest to some extent, as packages must be built, not just slurped down in wha…

I'm not sure about other container solutions, but Docker isn't a black box at all. It's fully introspectable -- I often dump a container's file system to see what's actually included.

Re: The sad state of sysadmin in the age of containers (2015)

#302

Post by hartzell »

dirkf wrote: post #180
ekidd wrote: post #144

Earlier quoted context omitted.

> without relying on any third-party servers being available and without pulling in dependencies that might have changed. There are two different issues here: 1. Not pulling in changed dependencies. This is what "lock files" are for: To limit builds to known version of every dependency. npm was terrible about this for a long time. Most other language package managers are better. 2. Not relying on third party servers…

At work we also cache all 3rd party dependencies locally. Not because the 3rd party servers might be down but to be sure that one/two/... years from now we can still recreate the same software we delivered at that time to a customer. Our build machines typically didn't even have internet access. If a dependency was not available offline (due to developer error or whatever) we would know quickly. In certain industry b…

> At work we also cache all 3rd party dependencies locally [...]

Can you say a bit about your platform and tooling? Are you working in a single language or a polyglot world? Is the cacheing at the network level or are your build tools aware of your mirrors?

I work in a space (biotech/pharma/...) that shares these concerns. I've solved the Perl specific version with Pinto (https://metacpan.org/pod/Pinto) and the more generic version with Spack (https://spack.io) [which is neat also because it supports installing multiple versions of applications, doesn't require root, ].

Re: The sad state of sysadmin in the age of containers (2015)

#303

Post by weberc2 »

glglwty wrote: post #299
weberc2 wrote: post #95

Earlier quoted context omitted.

I would say the main issue is that Nix is super difficult and requires you to intimately know your entire dependency tree down into Linux particularities. JavaScript makes matters worse, but even packaging a nontrivial Python package pulled directly from Pypi is often difficult. I want to like Nix, but it is far too pedantic to be practical. And this isn’t even mentioning the usability issues.

I think you only need to specify your immediate dependencies in nix?

Someone has to write the package definitions for those dependencies, and the public package repository doesn’t have broad coverage for many popular packages in many languages. This is understandable in that this is a massive effort, but that’s also the point—the effort to manage packages is huge and the extra effort compared to other tools is not a good bargain for many projects, nor is it much of consolation for project maintainers who would otherwise like to use it.

Re: The sad state of sysadmin in the age of containers (2015)

#304

Post by soraminazuki »

rumanator wrote: post #186
Zyst wrote: post #156

Earlier quoted context omitted.

Oh, check out Nix https://nixos.org/nix/ I use it for most of my dependencies, and software in OS X/Windows WSL/Linux It is a lovely environment to work with. Has completely changed how I feel about package management and the like. I know you were asking figuratively but something does happen to slot in that space.

> Oh, check out Nix Nix is a non-sequitur. Nix requires specific packages and package versions to be specified, and Nix requires the build process to be "free from side effects". That fails to address both problems stated by the OP, and actually try to reimplement what Debian's package building process already does. Other than trying to publicize Nix, your comment adds nothing to the discussion.

> That fails to address both problems stated by the OP

This is wrong and unfair. One of the major problems the OP has with Docker is that many software builds are unreproducible, encouraging many people to deploy binaries of dubious quality. Nix tries to solve precisely this problem through better tooling which makes it easy to ensure that its packages are reproducible and its dependencies easily verifiable. This is also what makes Nix distinct from Debian, which tries to improve package quality through policies and community collaboration.

> Nix requires the build process to be "free from side effects"

I disagree that this makes Nix irrelevant to the discussion at hand. Another gripe the OP had with Docker is that it sandboxes entire apps, making it a blackbox. Nix does sandboxing on a more granular level, which provides more transparency into individual packages.

Re: The sad state of sysadmin in the age of containers (2015)

#305

Post by finnthehuman »

opsthrowaway0 wrote: post #257
rb808 wrote: post #157

I agree the last decade has been really tough for sysadmins, but the payoff is developers have had a great time and I dont have to rely on those guys any more.

Flash bulletin from the engine room: you're still relying on "these guys" (and gals), you're just cost-sharing our salaries across whatever pile-o-garbage SaaS you convinced your architect to engage, including my own large cloud provider. Hint: it'd be more effective (and cheaper) to employ us directly, but you do you. That said, at the end of the day, I'm still sitting in my Aeron and your "great time" just means co…

All the claims that cloud computing is some boon to developers makes me suspect they might just work at a terribly organized company.

If you make developers pull double duty as sysadmins, then outsourcing surely makes things better for development. But it's still more work than handing the infrastructure and administration over to people that specialize in that. A well managed service is a ticket away, not getting someone who "knows docker pretty good" to have another background responsibility to neglect until it sets on fire.

Throughout the cloud boom I've worked at a company with a great IT department, not just a glorified helpdesk. The division makes sure that we provide a sane deliverable that doesn't need constant handholding and 10 hotfixes a day, and they give us the peace of mind that it's not going to go sideways. They sweat the details so much that the rare issues that have escalated to the head of IT and myself at the same computer investigating something were still relatively minor in the grand scheme of things.

Re: The sad state of sysadmin in the age of containers (2015)

#306

Post by rumanator »

lmm wrote: post #289
rumanator wrote: post #126

Earlier quoted context omitted.

> Well, clearly not, since you can't install Hadoop. You are complaining that you can't automatically install broken packages out of the box through the official repo. And packaging an application is the responsibility of the people working on that application, not the OS. What's your point?

An OS is a tool for running applications, not vice versa. Packaging an application should indeed be the responsibility of people working on that application, which is why language package managers like maven are successful. Debian explicitly rejected this philosophy in favour of one where OS maintainers are responsible for packaging all applications (thus the OS-specific packaging format and closed-world assumption o…

> Debian explicitly rejected this philosophy in favour of one where OS maintainers are responsible for packaging all applications

That assertion is quite wrong. Just because the OS has its official package repository, and just because OS maintainers volunteer their time to package some software projects, obviously that does not mean that maintainers are responsible for anything. You are confusing offering a convenience service with being responsible for packaging each and every software under the sun.

In fact, your baseless assertion ignores two basic facts: packages are proposed and adopted by volunteers, thus what you've described as "OS maintainers" is pretty much any random person who simply wants a software to be available for download in the distro's official repositories, and packaging systems such as Debian's apt supports private package repositories, where anyone can make available their packages to the world.

Another fact that you missed is that build systems such as Maven or msbuild or cmake or Gradle or whatever do support download packages only for a reason: convenience. It has absolutely nothing to do with where lies the responsibility of providing packages. It's convenient that a build fulfills all build dependencies. However, the responsibility of packaging and distributing packages of a software product lies exactly where it always was: those working on the software product.

Re: The sad state of sysadmin in the age of containers (2015)

#307

Post by [deleted] »

thyrsus wrote: post #292
AnIdiotOnTheNet wrote: post #245

Earlier quoted context omitted.

> The inability to install multiple versions is probably the only serious problem with the dpkg model. Also can't install to separate disks.

That's unlikely. Worst case: examine where the package will put its files, and prep your system with symbolic links to directories residing on a file system on the other disk(s). But that raises the question: why are your file systems sensitive to disk drives? Use a layer (or more) of indirection through LVM and RAID so to make the drive placement opaque and robust.

[deleted]

Re: The sad state of sysadmin in the age of containers (2015)

#308

Post by regularfry »

quicksilver03 wrote: post #259
regularfry wrote: post #46

Earlier quoted context omitted.

Debian is (mostly) low on the drama, but high on the expected package quality. It's not surprising they'd have problems here.

I couldn't disagree more, by looking at how badly Debian dares to diverge on basic packages such as Apache, nginx and exim it's clear to me that they don't care at all about quality.

Divergence from the upstream isn't a marker for or against quality, as Debian chooses to define it.

Re: The sad state of sysadmin in the age of containers (2015)

#309

Post by Conan_Kudo »

throwaway2048 wrote: post #212
briffle wrote: post #209

Earlier quoted context omitted.

you can use a tool like snap or appimage to pull down the newest full release of your software.. at least then, all the dependancies are 'bundled' into the image, and your not having people go randomly install crap from around the internet with no real way to update.

Building debian packages from source definitely does not involve "randomly installing crap from around the internet". The .sdeb format packages up sources and build scripts into one atomic, build-able unit.

The only .sdeb archive format I know of is the one for debbuild[1]. Most Debian package build mechanisms don't have a source package archive format.

[1]: https://github.com/debbuild/debbuild

Re: The sad state of sysadmin in the age of containers (2015)

#310

Post by AnIdiotOnTheNet »

thyrsus wrote: post #292
AnIdiotOnTheNet wrote: post #245

Earlier quoted context omitted.

> The inability to install multiple versions is probably the only serious problem with the dpkg model. Also can't install to separate disks.

That's unlikely. Worst case: examine where the package will put its files, and prep your system with symbolic links to directories residing on a file system on the other disk(s). But that raises the question: why are your file systems sensitive to disk drives? Use a layer (or more) of indirection through LVM and RAID so to make the drive placement opaque and robust.

> why are your file systems sensitive to disk drives?

A couple of systems I have at home have a 16GB primary disk, so it is useful to be able to put things on an external disk. I have yet to see any package manager that can actually handle that.

> Use a layer (or more) of indirection through LVM and RAID...

Yeah, that's a great idea, just introduce more complication and abstraction to make up for a shortcoming in how Linux developers think about applications.

Or, just have your programs be self contained in a single file or folder and put them wherever it makes sense.

Post reply on HN