Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sad state of sysadmin in the age of containers (2015)

vitavonni.de

Post reply on HN
Print view View original

281–290 of 324 posts

Re: The sad state of sysadmin in the age of containers (2015)

#281

Post by oblio »

eeZah7Ux wrote: post #241
oblio wrote: post #224

Earlier quoted context omitted.

While you could argue about: > install packages for a single user because of the security angle, Not allowing this: > install multiple versions of the same package is completely indefensible. Who are you, package manager, to decide that I shouldn't be able to use different versions of the same package? Do you know more about my context than I do?

> Who are you, package manager, to decide that I shouldn't be able to use different versions of the same package? That's one of main point of a distribution , rather that simply throwing software on a hard drive. The maintainers have to work hard to guarantee that a specific set of packages and version work well together. Well enough for most users to deploy 99% of the packages without surprises. And then backport se…

> Given your statements about distributions... most likely yes.

Hello there, ad hominem.

Linux distributions are overreaching. They should offer those services for core software. I should be able to install Inkscape 0.91 and 0.92 at the same time, easily.

Re: The sad state of sysadmin in the age of containers (2015)

#282

Post by adamc »

buzzkillington wrote: post #25

Looked at the Debian wiki the OP linked to: >Debian currently does not include Hadoop packages. There are a number of reasons for this; in particular the Hadoop build process will load various dependencies via Maven instead of using distribution-supplied packages. Java projects like this are unfortunately not easy to package because of interdependencies; and unfortunately the Hadoop stack is full of odd dependencies…

That triggered memories... happier in Python.

Re: The sad state of sysadmin in the age of containers (2015)

#283

Post by raquo »

apta wrote: post #177
buzzkillington wrote: post #117

Earlier quoted context omitted.

Something Java and the Java ecosystem is terrible at. I've had project where we needed all of 7,8,9 and 10 [0] for the data stack: Hadoop, Scala, Spark, HDFS, zookeeper, pyspark. Good luck setting that up on a single machine, containers all the way. The java path nightmares I still have. I'm probably one of the few people in the word who know this much about the JVM without ever having written any Java. [0] We should…

> I've had project where we needed all of 7,8,9 and 10 [0] for the data stack: Hadoop, Scala, Spark, HDFS, zookeeper, pyspark. I suspect that a big reason was due to using Scala, which is notorious for not maintaining binary backward compatibility across minor releases.

Scala's version is epoch.major.minor, not major.minor.patch. I though 2.12.(x+1) is generally binary compatible with 2.12.x. Or at least I personally never ran into that issue with Scala itself.

Re: The sad state of sysadmin in the age of containers (2015)

#284

Post by edoo »

u801e wrote: post #244
chefandy wrote: post #198

Earlier quoted context omitted.

I've worked with developers that have used git for 10 years who didn't fully realize what all of the 'git reset' options entailed, and I don't blame them. Git is complicated and you could certainly have a perfectly effective workflow with it for your whole career without using most of the features. If you'd only worked in environments in which code was entirely managed in GitHub, you probably wouldn't know that eithe…

> I've worked with developers that have used git for 10 years who didn't fully realize what all of the 'git reset' options entailed While knowing some of them, if you use git reset on a frequent basis is expected, knowing where to find information on the other options is essential. I frequently go back and read through the man pages for various git commands so that I understand what will happen if I use a particular…

I give this to my devs: https://www.atlassian.com/git/tutorials/atlassian-git-cheats...

Re: The sad state of sysadmin in the age of containers (2015)

#285

Post by lmm »

notacoward wrote: post #166
lmm wrote: post #77

You've got to look at this in a context where platform package managers like apt are simultaneously 1) platform-specific 2) jealous, insisting that every language has to conform to their way of doing things and 3) fundamentally not very good, having very limited ability to do things like install packages for a single user or install multiple versions of the same package. Platform package managers like maven have been…

Language-specific package managers are also "jealous" as you put it. They insist that every platform conform to their way of doing things. They're also promiscuous, in the sense that most have really poor dependency management ("let's build the internet today just in case") and practically no security. The computing world would be a better place if the people forcing their language-specific idioms on the rest of the…

> They're also promiscuous, in the sense that most have really poor dependency management ("let's build the internet today just in case") and practically no security.

Not criticisms that apply to maven. Binary dependencies on released versions are the norm, and all packages in the central repository are signed with GPG, just as with apt.

> The computing world would be a better place if the people forcing their language-specific idioms on the rest of the world had instead spent their time improving the platform package managers that already existed at the time.

If those platform package managers had been open to being improved, which I don't think they were (and conversely, deb/apt originated because Debian insisted on doing things their own way rather than improving RPM). Apt's closed-world assumptions seem like a policy decision. Apt not being cross-platform is definitely policy. And surely it's occurred to people in Debian that the ability to have per-user installs, and more importantly non-shared/non-overlapping installs of libraries that are depended on, would be useful; the insistence on single system-wide installs of libraries can only be a policy decision, and one that's turned out to result in a less usable system.

Re: The sad state of sysadmin in the age of containers (2015)

#286

Post by lmm »

rumanator wrote: post #121
lmm wrote: post #77

You've got to look at this in a context where platform package managers like apt are simultaneously 1) platform-specific 2) jealous, insisting that every language has to conform to their way of doing things and 3) fundamentally not very good, having very limited ability to do things like install packages for a single user or install multiple versions of the same package. Platform package managers like maven have been…

> jealous, insisting that every language has to conform to their way of doing things Let's ignore for a moment all your baseless assertions and focus on the following question: how do you ensure that a build is reproducible if you have no control on which required dependencies are downloaded by a build system which requires root access? Is jealousy the only explanation that comes up to you?

Reproducible builds were a non-goal for apt for most of its existence.

Re: The sad state of sysadmin in the age of containers (2015)

#287

Post by lmm »

apta wrote: post #177
buzzkillington wrote: post #117

Earlier quoted context omitted.

Something Java and the Java ecosystem is terrible at. I've had project where we needed all of 7,8,9 and 10 [0] for the data stack: Hadoop, Scala, Spark, HDFS, zookeeper, pyspark. Good luck setting that up on a single machine, containers all the way. The java path nightmares I still have. I'm probably one of the few people in the word who know this much about the JVM without ever having written any Java. [0] We should…

> I've had project where we needed all of 7,8,9 and 10 [0] for the data stack: Hadoop, Scala, Spark, HDFS, zookeeper, pyspark. I suspect that a big reason was due to using Scala, which is notorious for not maintaining binary backward compatibility across minor releases.

Scala breaks binary compatibility approximately every 2-3 years, which is fast by JVM standards but slow compared to most programming languages (e.g. Python releases an incompatible version every 1-2 years). You can blame Scala for at most one incompatibility in that long list; older versions of Scala are incompatible with Java 9+ (this is true for many JVM frameworks too, as Java 9 made breaking changes).

Re: The sad state of sysadmin in the age of containers (2015)

#288

Post by lmm »

mcguire wrote: post #266
lmm wrote: post #77

You've got to look at this in a context where platform package managers like apt are simultaneously 1) platform-specific 2) jealous, insisting that every language has to conform to their way of doing things and 3) fundamentally not very good, having very limited ability to do things like install packages for a single user or install multiple versions of the same package. Platform package managers like maven have been…

But platform managers like maven are 1) language-specific, 2) jealous, insisting that every OS and extra-linguistic dependency has to conform to their way of doing things, and 3) fundamentally, not very good, particularly with regard to multiple versions of the same package (technically, OSGi could do that...), repeatable builds, and the desire for do-what-I-mean behavior. Platform managers have been successful becau…

> 2) jealous, insisting that every OS and extra-linguistic dependency has to conform to their way of doing things

Not really. Language-specific package managers are usually capable of integrating with "system" dependencies from outside the language (certainly maven has decent support for this) - they kind of have to be after all.

> 3) fundamentally, not very good, particularly with regard to multiple versions of the same package (technically, OSGi could do that...), repeatable builds, and the desire for do-what-I-mean behavior.

How so?

Re: The sad state of sysadmin in the age of containers (2015)

#289

Post by lmm »

rumanator wrote: post #126
lmm wrote: post #89

Earlier quoted context omitted.

Well, clearly not, since you can't install Hadoop. Before you blame that on Hadoop, remember that it only requires some really quite basic things from its package manager, which apt is nevertheless completely unable to do: use libraries in the normal recommended way, play nice with Java, work cross-platform.

> Well, clearly not, since you can't install Hadoop. You are complaining that you can't automatically install broken packages out of the box through the official repo. And packaging an application is the responsibility of the people working on that application, not the OS. What's your point?

An OS is a tool for running applications, not vice versa. Packaging an application should indeed be the responsibility of people working on that application, which is why language package managers like maven are successful. Debian explicitly rejected this philosophy in favour of one where OS maintainers are responsible for packaging all applications (thus the OS-specific packaging format and closed-world assumption of their package manager).

Re: The sad state of sysadmin in the age of containers (2015)

#290

Post by celticmusic »

Spivak wrote: post #152
celticmusic wrote: post #135

Earlier quoted context omitted.

You have make call the platform specific build systems.

What's the point in a project that only uses a single build system? You're essentially getting nothing out of make except indirection. Might as well just have a build.sh at the root of the repo.

familiarity, documenting the process for building, and last but not least, access to the entire unix toolchain.
Post reply on HN