Live data from Hacker News

The sad state of sysadmin in the age of containers (2015)

vitavonni.de

251–260 of 324 posts

Re: The sad state of sysadmin in the age of containers (2015)

#251
post #144
post #19

I recently had a similar discussion with people using npm for building a CSS framework library. I tried to explain the concept of getting a pre-downloaded tarball and using "make" (or similar) to produce target artifacts from source files in a deterministic, repeatable and reliable manner, without relying on any third-party servers being available and without pulling in dependencies that might have changed. It seems…

> without relying on any third-party servers being available and without pulling in dependencies that might have changed. There are two different issues here: 1. Not pulling in changed dependencies. This is what "lock files" are for: To limit builds to known version of every dependency. npm was terrible about this for a long time. Most other language package managers are better. 2. Not relying on third party servers…

There are other advantages such as build speed. Having local copies is simply faster.

Re: The sad state of sysadmin in the age of containers (2015)

#252
post #36

Looked at the Debian wiki the OP linked to: >Debian currently does not include Hadoop packages. There are a number of reasons for this; in particular the Hadoop build process will load various dependencies via Maven instead of using distribution-supplied packages. Java projects like this are unfortunately not easy to package because of interdependencies; and unfortunately the Hadoop stack is full of odd dependencies…

Who still uses Hadoop anyway? https://spark.apache.org/ https://www.iguazio.com/data-science-post-hadoop/

Sounds like this is the root issue, and not significantly about package managers.

No one is willing to invest the time to untangle the build process and fix compat issues in the software. The project is slowly dying out.

Re: The sad state of sysadmin in the age of containers (2015)

#253
I don't agree.

Systems are more complicated to build with a lot of dependencies that's true, but this is mitigated by shipping pre-built binaries and containers which have the dependencies bundled. Yes, those binaries can be from untrusted sources but if you use those you're taking a risk - the same as if you use code from an untrusted source (just because it's not compiled doesn't make it safe, when was the last time your organisation audited anything), ultimately a business should be using banaries from businesses they trust.

Containers actually increase security in the following ways: 1. They run isolated from the host system and if you set things up properly they should be non root. 2. They're immutable and cannot be tampered with. 3. Because they work on layers you can change them without affecting the underlying container, to upgrade you can just replace the layer that you inherit from, simplifying the upgrade path (and being immutable, stateless, and versioned there's no risk to upgrading, you can very easily rollback). 4. In the example listed above like Haddoop you can install the outdated version of Java that it depends on without compromising the host or other workloads on that server.

Of course, this all breaks down if you don't use containers from trusted sources and if you don't have processes in place.

It works in a modern environment with pipelines and automation, it doesn't work as well in a more traditional environment with sysadmins hand wrangling deployments.

Of course the most secure way to do things is to have deterministic builds of a codebase which creates a hash which you can compare with a canonical source, but this is tricky and not all build toolchains support this properly so you'll end up with mixed results. For a streamlined business the best compromise is to get applications from trusted vendors and rely on static analysis and the ability to quickly upgrade if a vulnerability is discovered.

Ultimately, the one of the best ways to secure a system is to establish an unbroken, audited, and automated trust chain and remove the ability for human intervention.

Re: The sad state of sysadmin in the age of containers (2015)

#254
post #128

Earlier quoted context omitted.

> They could only do it for a handful of servers, change request took months and the systems were not to be 'touched'. Those old ways of doing things were indeed slower than they needed to be in many cases, and I'm glad that we sped them up. I don't believe that we needed to throw the baby away with the bathwater though. The problem now is that we build systems which are unmanageable and unmanaged. 'Throw it in a con…

After a few decades it's no longer a baby ;) Software is becoming more complex and moving faster than ever. 'Controlling your dependencies' was always an illusion and a trade-off at best. Often a trade-off against security, inheriting the folder of JAR files from your predecessor and such. I think of containers and all these new tools as designs which are supposed to help us manage existing complexity. They don't cre…

I think a lot of folks who have been through a few tech cycles just see us building the same tools over and over again adding complexity to the overall system.

I think the main cause here is, ironically, the fact that businesses keep trying to remove sysadmin (as a discipline) from the IT value chain and replace it with subscription services. Sure, you can do it, but at some point things will break and you need someone who understands what is happening at the lowest levels of abstraction.

Re: The sad state of sysadmin in the age of containers (2015)

#255
post #201

Earlier quoted context omitted.

> A lot of them involve messy build steps, including downloading binaries or source tarballs without verification Do they though? Whenever I write Dockerfiles that depend on external downloads, I always check the hash matches one baked into the Dockerfile itself, and I've always seen others doing the same.

I have seen this many times, generally the high profile containers do that, and these issues are more common among my peers. This requires self discipline and containers were pushed as a panacea for environments where the discipline is lacking.

Do you have any examples of high-profile images that do this?

I really just haven't seen this with the ones I use.

Re: The sad state of sysadmin in the age of containers (2015)

#256

Earlier quoted context omitted.

Language-specific package managers are also "jealous" as you put it. They insist that every platform conform to their way of doing things. They're also promiscuous, in the sense that most have really poor dependency management ("let's build the internet today just in case") and practically no security. The computing world would be a better place if the people forcing their language-specific idioms on the rest of the…

So, what would 'pip install s3cmd' look on Windows or macOS?

What is the point of your question?

Re: The sad state of sysadmin in the age of containers (2015)

#257
post #157

I agree the last decade has been really tough for sysadmins, but the payoff is developers have had a great time and I dont have to rely on those guys any more.

Flash bulletin from the engine room: you're still relying on "these guys" (and gals), you're just cost-sharing our salaries across whatever pile-o-garbage SaaS you convinced your architect to engage, including my own large cloud provider. Hint: it'd be more effective (and cheaper) to employ us directly, but you do you.

That said, at the end of the day, I'm still sitting in my Aeron and your "great time" just means concentrating an entire discipline of professionals into cloud companies and ISPs. In this dream scenario (for who?), you get to sit around waiting for ticket updates while your revenue craters instead of picking up the phone or using that hole on your face to communicate with a peer. I get to focus on systems at scale and on monitoring and metrics instead of an annoying developer who escalates to their SVP and makes an inter-org ruckus when their TLS certificate expires and they can't find the error in their voluminous Javascript "logs". To be clear: I'm having a great time. There's a suspicious TAM in between me and you now, and you don't have my extension. I prayed for this day across many developer escalations and still can't believe it's here.

Seriously, people who are likely replaceable with Microsoft Access and a few Thoughtworks clones shouldn't throw stones at the professions completely supporting their Blue Bottle habit.

Oh yeah, the Internet keeps on quietly working, too. Tip jar is to your left.

Re: The sad state of sysadmin in the age of containers (2015)

#258
post #89

Earlier quoted context omitted.

Well, clearly not, since you can't install Hadoop. Before you blame that on Hadoop, remember that it only requires some really quite basic things from its package manager, which apt is nevertheless completely unable to do: use libraries in the normal recommended way, play nice with Java, work cross-platform.

> Well, clearly not, since you can't install Hadoop. You are complaining that you can't automatically install broken packages out of the box through the official repo. And packaging an application is the responsibility of the people working on that application, not the OS. What's your point?

For Linux distros, the primary responsibility for packaging is actually with the distro maintainers, since packaging is distro-specific.

That said, a well-written app should not present any difficulty for the package maintainer.

Re: The sad state of sysadmin in the age of containers (2015)

#259

Earlier quoted context omitted.

>Unfortunately, the build process for these packages is currently of a disastrous quality, and should only be attempted within disposable virtual machines, as it requires root permissions and will install non-packaged software. https://wiki.debian.org/Hadoop If Debian, a distro on the lower side of dramatic, calls it disastrous, it's pretty bad.

Debian is (mostly) low on the drama, but high on the expected package quality. It's not surprising they'd have problems here.

I couldn't disagree more, by looking at how badly Debian dares to diverge on basic packages such as Apache, nginx and exim it's clear to me that they don't care at all about quality.

Re: The sad state of sysadmin in the age of containers (2015)

#260

Earlier quoted context omitted.

> Well, clearly not, since you can't install Hadoop. You are complaining that you can't automatically install broken packages out of the box through the official repo. And packaging an application is the responsibility of the people working on that application, not the OS. What's your point?

For Linux distros, the primary responsibility for packaging is actually with the distro maintainers, since packaging is distro-specific. That said, a well-written app should not present any difficulty for the package maintainer.

> For Linux distros, the primary responsibility for packaging is actually with the distro maintainers, since packaging is distro-specific.

No it is not. Although distros might pick and adopt some packages and include them in their official repository, it's obvious to anyone that as a distro maintainer you are not responsible for packaging each and every software package under the sun.

Moreso, linux distros such as Debian rely on volunteers to propose and adopt packages, which obviously also includes people affiliated with each software project.

Additionally, it's patently obvious that the responsibility of packaging and making a software available to the public lies on the people involved in developing the software project.

> That said, a well-written app should not present any difficulty for the package maintainer.

That is true, particularly as the primary package maintainers are in fact those actually developing the software project.

Post reply on HN