Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sad state of sysadmin in the age of containers (2015)

vitavonni.de

Post reply on HN
Print view View original

191–200 of 324 posts

Re: The sad state of sysadmin in the age of containers (2015)

#191

Post by buzzkillington »

boxy310 wrote: post #155
itg wrote: post #67

Earlier quoted context omitted.

And then add PySpark on top of that. Couldn't leave my last job fast enough when they decided to use Hadoop/PySpark when the largest incoming files we received were at most a few GBs.

I once had a consulting gig where the customer desperately wanted to build a Spark/Scala ML pipeline, for a dataset that was 10 MB. We spent 3 months hammering it together for a flat Python process that would've taken us 2 weeks.

If you'd sent it off to mechanical Turk it would have been done in an afternoon.

Re: The sad state of sysadmin in the age of containers (2015)

#192

Post by yori »

sudosysgen wrote: post #123
yori wrote: post #102

Earlier quoted context omitted.

What difference does it make if you run `make install` or `curl | bash`? In both cases, you are running code you have not audited yourself. Or are you the kind who inspects every Makefile and installer script before installing? If not, why is one better than the other?

Because at least with make install you can have some trust in the delivery method, ie encrypted git or whatever. If you use just plain http....

So wget https://example.com/installer.tar.gz; tar -xvzf installer.tar.gz; cd installer; make install is okay for you?

But curl https://example.com/installer.sh | bash is not? Why?

Re: The sad state of sysadmin in the age of containers (2015)

#193

Post by winrid »

jwr wrote: post #19

I recently had a similar discussion with people using npm for building a CSS framework library. I tried to explain the concept of getting a pre-downloaded tarball and using "make" (or similar) to produce target artifacts from source files in a deterministic, repeatable and reliable manner, without relying on any third-party servers being available and without pulling in dependencies that might have changed. It seems…

Easiest thing to do to solve this would be to have something like artifactory or even squid in front of npm and use exact version numbers. Front end devs are going to use NPM. Solve the other problems separately like you would with any other build system.

Re: The sad state of sysadmin in the age of containers (2015)

#194

Post by eckza »

winrid wrote: post #193
jwr wrote: post #19

I recently had a similar discussion with people using npm for building a CSS framework library. I tried to explain the concept of getting a pre-downloaded tarball and using "make" (or similar) to produce target artifacts from source files in a deterministic, repeatable and reliable manner, without relying on any third-party servers being available and without pulling in dependencies that might have changed. It seems…

Easiest thing to do to solve this would be to have something like artifactory or even squid in front of npm and use exact version numbers. Front end devs are going to use NPM. Solve the other problems separately like you would with any other build system.

Sure. And then when something unexpected happens, you have two problems.

Re: The sad state of sysadmin in the age of containers (2015)

#195

Post by r3trohack3r »

lmm wrote: post #77

You've got to look at this in a context where platform package managers like apt are simultaneously 1) platform-specific 2) jealous, insisting that every language has to conform to their way of doing things and 3) fundamentally not very good, having very limited ability to do things like install packages for a single user or install multiple versions of the same package. Platform package managers like maven have been…

This is a very negative view.

Once you get used to this method of packaging it’s actually really empowering for the end user. You can download the source for anything in the system, modify it, recompile, and install it all with a standard set of commands (no one-off custom build steps for every package).

I feel it’s the closest thing to the spirit of “free software” we have today: software designed to empower the end user to read, understand, and modify their system without a huge amount of obfuscation.

It increases the burden on the packagers to reduce the burden on the users. I would be interested to see how often these features actually get used though. I wish distributions would put this front and center.

https://wiki.debian.org/WhyDebian

Re: The sad state of sysadmin in the age of containers (2015)

#196

Post by _-o-_ »

doomslice wrote: post #133
weberc2 wrote: post #93

Earlier quoted context omitted.

There’s a pretty straightforward solution to this problem which is to run your own NPM server even if it’s only a caching proxy. For some orgs, the uptime provided by the third party servers is sufficient.

Having been at 3 companies that did this, I can say with certainty that our caching proxies (artifactory in all of them) were much more likely to go down than the public repos were.

Yeah, I think I've lost a day in total when our local npm was finicky on several occasions or failed to fetch latest version of a package that we've developed in house, but made publicly available.

Re: The sad state of sysadmin in the age of containers (2015)

#197

Post by ascotan »

This post reminded me of an old article: https://blog.codinghorror.com/vampires-programmers-versus-we...

The big win for 'containers' is that they separate the concerns of sysadmins from developers. The days of the sysadmin administering the tomcat and apache installs are shortly coming to an end. For my part, i think this is a good thing because devs and sysadmins have different concerns that don't overlap all the time.

Re: The sad state of sysadmin in the age of containers (2015)

#198

Post by chefandy »

phaedrus wrote: post #167
y0y wrote: post #69

Earlier quoted context omitted.

I know you're taking some flack for making it about age, but I do think there's some merit there. I'm 34. I feel like I'm in the middle in terms of software developer / sysadmin age. Lots of bright minds came before me and there are lots of bright minds out there right now in their early 20s. A key difference is I can remember a time when network connectivity was flaky. When it was hardly a given. Even when it was av…

I'm 36 and recently was assigned to mentor a new employee in his 20s. We had a moment of miscommunication when I asked him to use git to clone a local repository. He was confused when he couldn't find it on github.com (what I'd sent him was a path to our private network share). I had to explain that, yes you can use the github.com client if you want, but "git" is different from "github". I honestly couldn't tell if h…

I've worked with developers that have used git for 10 years who didn't fully realize what all of the 'git reset' options entailed, and I don't blame them. Git is complicated and you could certainly have a perfectly effective workflow with it for your whole career without using most of the features. If you'd only worked in environments in which code was entirely managed in GitHub, you probably wouldn't know that either. Judging someone because they don't possess the same slices of implementation-specific knowledge you do doesn't really make sense. They almost certainly know things that you don't simply because you've never encountered situations in which you had to learn about them, or for that matter, remember them even if you did.

Re: The sad state of sysadmin in the age of containers (2015)

#199

Post by snaky »

boxy310 wrote: post #155
itg wrote: post #67

Earlier quoted context omitted.

And then add PySpark on top of that. Couldn't leave my last job fast enough when they decided to use Hadoop/PySpark when the largest incoming files we received were at most a few GBs.

I once had a consulting gig where the customer desperately wanted to build a Spark/Scala ML pipeline, for a dataset that was 10 MB. We spent 3 months hammering it together for a flat Python process that would've taken us 2 weeks.

> This find xargs mawk pipeline gets us down to a runtime of about 12 seconds, or about 270MB/sec, which is around 235 times faster than the Hadoop implementation.

https://adamdrake.com/command-line-tools-can-be-235x-faster-...

Re: The sad state of sysadmin in the age of containers (2015)

#200

Post by jeltz »

gerbilly wrote: post #114
adgasf wrote: post #71

Earlier quoted context omitted.

Makefiles are too low level to be convenient for anything other than simple projects on one platform.

If the alternative you are hinting at is CMake, I can't really agree with you. I tried it recently and was shocked at how awful the syntax and usability was. It's begging for a tool to auto-detect your dependencies from source. I was trying to port an existing project to it and I gave up when I realised that I'd have to manually create about 30 CMakeList.txt files or whatever they call them. Meanwhile during the same…

Take a look at Ninja and Bazel. I ahree with you that CMake is worse than autoconf but I would argue that the tools which came after CMake are better than CMake and autoconf. So while there certainly are regressions not every new generation is worse than the previous.
Post reply on HN