The sad state of sysadmin in the age of containers (2015) • Hacker News Board

Re: The sad state of sysadmin in the age of containers (2015)

#301

Post by zapita » Wed, May 16, 2018, 9:51 PM UTC

dredmorbius wrote: post #291

zapita wrote: post #276

Earlier quoted context omitted.

> Modern devops, with its million tools that break backward compatibility every month sometimes becomes the running joke at lunch. Ironically, modern devops and its million broken tools are a primary source of revenue for cloud providers, helping pay for your lunch in the first place.

Risk arbitrage. Present capability vs. future catastrophic risk. Technology is debt.

I honestly can't tell whether you're making a sincere but unintelligible argument, or just trolling.

Do you disagree with what I said?

Re: The sad state of sysadmin in the age of containers (2015)

#302

Post by dozzie » Wed, May 16, 2018, 9:51 PM UTC

noir_lord wrote: post #260

ams6110 wrote: post #248

Earlier quoted context omitted.

Ansible is one "ohh shiny" thing that has greatly increased my productivity as a sysadmin. Before that I would automate what I could with ssh and pdsh and scripts, but it was never as well polished as Ansible. I'm even using ansible for ad-hoc stuff (tweaking a config, restarting a service) because it's easier to do that from a management server than log in to some remote host, get oriented as to the OS distribution…

I like ansible with vagrant as well - it makes for a nice clean way of deploying to development environments while also been nicely 'self' documenting and not limiting (you can drop back to shell), it's a lovely tool for the most part. Edit: The thing I really like about Ansible is how unsexy it is, it's just a nice sane way of doing largely what you could do yourself with ssh and bash but in a language that doesn't…

It would be even nicer if people weren't advocating it as a configuration management system, only as what it is: a deployment system.

Re: The sad state of sysadmin in the age of containers (2015)

#303

Post by disordinary » Wed, May 16, 2018, 9:52 PM UTC

There's always been propriety code installed on systems, who knows what that big Oracle database is actually doing?

Are these sysadmins actually looking through the open source code that they're compiling to make sure that there isn't a security flaw in plain sight?

With Docker you can get containers maintained by a trusted source, as a sysadmin you don't have to deal with all the hassle of upgrading things, you can just replace the container with the latest version. With Docker content trust the containers are signed.

There have always been cowboys out there and that has nothing to do with the tech stack.

Re: The sad state of sysadmin in the age of containers (2015)

#304

Post by commandlinefan » Wed, May 16, 2018, 9:58 PM UTC

ironjunkie wrote: post #223

commandlinefan wrote: post #214

Earlier quoted context omitted.

I've seen this brewing for a while, and getting worse and worse. Back in the 80's and 90's, there were developers who would code their own sorting or hashing routines rather than linking in some external library to handle this "solved" problem. The perjorative term "Not Invented Here" (NIH) grew to describe those developers and they were shamed into reusing code whenever there was code to reuse. And in some cases (li…

Interesting take on the NIH term. I thought this was more an ego thing for the big tech companies. They love to reinvent existing things to look like geniuses

I think you're correct on the origin of the term, and I should have mentioned that - but in the past few decades, I've been accused of "NIH"-ing whenever I've "rolled something" of my own from authentication to IoC. Just because there's a library that has a particular description attached to it doesn't mean that it should be used as often as it can be.

Re: The sad state of sysadmin in the age of containers (2015)

#305

Post by dozzie » Wed, May 16, 2018, 10:00 PM UTC

user5994461 wrote: post #280

virgilp wrote: post #259

Earlier quoted context omitted.

I know our SREs once screwed the config for sshd, and considered themselves very lucky that they had puppet on the machines and could push a fixed configuration (if they had used exclusively ansible, that'd be the end of it - no way to connect or to deploy new configuration) [edit] To clarify - ansible is great, and we use it. Just saying that, as everything, it still has (sometimes subtle) downsides in various scena…

Well, teach your sysadmin to use the system configuration tester when they edit a system configuration file. Nothing to do with ansible really, except that ansible allows to prevent that easily.

> Well, teach your sysadmin to use the system configuration tester when they edit a system configuration file.

Wrong. Teach your sysadmin not to overload a single service with different functions (debugging channel, user-facing shell service, running remote commands, file upload, and config distribution channel), especially not the one that should not be used in batch mode, without human supervision.

When you write an application, you don't put a HTTP server in database connection handling code, but when it comes to server management, suddenly the very same approach is deemed brilliant, because you don't run an agent (which is false, because you do, it's just not a dedicated agent).

Re: The sad state of sysadmin in the age of containers (2015)

#306

Post by zippitydoodah68 » Wed, May 16, 2018, 10:06 PM UTC

eropple wrote: post #161

mcguire wrote: post #148

Earlier quoted context omitted.

Speaking as an (ex- (sort of)) sysadmin, " All of which are clunky, opaque, and harder to work with than any other language I have used in the last decade. " Wrong question. A better one: Is it more clunky, opaque, and harder to work with than every other language that's appeared in the last decade? Because no one seems to agree on what is specifically better.

I disagree that it is the wrong question. "Every other language" doesn't matter because I don't value homogeneity and I think that homogeneity of programming language is a fool's errand. I am comfortable shipping production code in most of the languages in current use; in my estimation, none of the major build systems out there are as opaque or difficult to use correctly as make/shell. (And I am what a current sysadm…

And you are definitely not a sysad or any sort of sysadmin. The core mission of a sysad is to build the best environment possible while restricting that environment.

You seem to be the sort of developer that developers love and the sort of sysadmin that gets fired in the first week Just my .02 after two and a half decades.

Not to say that you are entirely wrong or your approach doesn't have merit in the new world (especially SV). But it doesn't work for sysadmins and production environments anywhere but your bubble. Not yet.

Re: The sad state of sysadmin in the age of containers (2015)

#307

Post by spookthesunset » Wed, May 16, 2018, 10:09 PM UTC

zzzeek wrote: post #234

drbawb wrote: post #138

Earlier quoted context omitted.

>Don't build your containers from public, un-trusted images! The author is complaining that you can't build these private trusted images. Software developers have got it in their head that containers are a way to package & distribute software. They're not, that's what the OS's package managers are for. If your software requires Docker as a build dependency, you have failed to properly package your software. As a conc…

This comment makes way more sense to me that the original blog post. Yes, nobody should be relying upon docker as their distribution platform. That's pretty terrible. Ubiquity I've observed seems pretty uncomfortable just supporting the major distros, I actually wrote some docker stuff to pull down their .debs, crack them open and install the binaries inside on a fedora/centos system. That's closed source for ya.

> I actually wrote some docker stuff to pull down their .debs, crack them open and install the binaries inside on a fedora/centos system.

Why would you want to do that though? Treat the whole thing as a black box running inside docker and be done with it. The second you crack it open, you get to support it. Let Ubiquiti support it, after all that's what you are paying them the big bucks for.

Re: The sad state of sysadmin in the age of containers (2015)

#308

Post by dozzie » Wed, May 16, 2018, 10:14 PM UTC

kokey wrote: post #230

user5994461 wrote: post #176

Earlier quoted context omitted.

CFEngine is basic text manipulation, it's not comparable to the rest. Puppet and Chef was the first generation. I wouldn't recommend. All the companies and people I know using Chef migrated away from it after many disasters. Nowadays, it's only mentioned in interviews to find out if candidates have real world fire fighting experiences. Ansible is good. Used that for managing hundreds of machines at multiple jobs (som…

CFengine, at least version 3, was probably the furthest away from string manipulation (and I was given the impression that text file content manipulation was considered a bad idea with it). What killed it was promise theory, which is actually a great theory and works quite well but made writing the bundles painfully hard and also hard to maintain. Also during the early days of v3 it was probably lacking a ton of esse…

> This leads me to another thing that bothers me and that is where people think it's a situation of having to use e.g. Puppet or Ansible as if they're conflicting choices for the same tasks.

That's mainly because Ansible folks advertise it as a configuration management tool, while in fact it's a deployment tool. The former needs asynchronous operations, especially because a node that is supposed to be reconfigured can be temporarily down. The latter needs to be executed synchronously, with reports being read as they come by an operator.

There are several other operation modes that are useful for a sysadmin, like running a predefined procedure with parameters supplied from the client, or running a one-off command everywhere (even on the servers that are currently down, as soon as they are up), but we don't have many tools to cover those cases.

Re: The sad state of sysadmin in the age of containers (2015)

#309

Post by cookiecaper » Wed, May 16, 2018, 10:17 PM UTC

pvg wrote: post #294

cookiecaper wrote: post #251

Earlier quoted context omitted.

> Any separation is ineffective except along this one particular completely arbitrary dividing line? If that were true we'd still be hunting and gathering and nothing else. Hardly arbitrary -- hardware is fixed at the time of manufacture. Hardware engineers should be well-acquainted with software concerns and needs, but the years-long feedback cycle and real expenses associated with hardware development creates a nat…

It's arbitrary when you claim there are no sensible divisions in software. I think your entire lengthy argument is a sort of elaborate fantasy about how much better the world would be if everyone was just like you or at least, just like you imagine yourself to be. It's fun but not a particularly realistic or constructive way to look at the world.

> It's arbitrary when you claim there are no sensible divisions in software.

It's about the fungibility of the problem space. I don't know how you expect your core team to make reasonable decisions about the tradeoffs if they a) don't understand more than one of the platform elements; and/or b) don't have any responsibility or accountability for the tradeoffs that get made, because now it's another segment's problem. Indeed, when I've been on teams primarily comprised of non-generalists, these decisions were almost always a matter of bureaucracy and politics.

> I think your entire lengthy argument is a sort of elaborate, lengthy fantasy about how much better the world would be if everyone was just like you or at least, just like you imagine yourself to be.

I've worked on teams that were mostly "generalist" and teams where the "generalist" type was either absent or artificially constrained. My perspectives are drawn from those experiences, and have developed based on a hard-earned worldview that says people reliably act in favor of their own expedience. Doesn't seem very fantastic to me. ¯\_(ツ)_/¯

Re: The sad state of sysadmin in the age of containers (2015)

#310

Post by dozzie » Wed, May 16, 2018, 10:19 PM UTC

dba7dba wrote: post #254

brightball wrote: post #160

Earlier quoted context omitted.

Would be interested to get your opinion on Puppet/Ansible/Chef/CFEngine/SaltStack

+1 for Ansible from me. Of all the tools, I first heard of Puppet first and so I'm assuming it was first on scene? From my limited experience, it seems Puppet is most widely used tool because of that reason. Not necessarily the best of the bunch, but first on the scene. Considering the effort required to roll it out, I am assuming whatever is deployed first will stay as the tool of choice. I've tried out Puppet, Salt…

> [...] the actual change can happen on the "client servers" at any point within next 20 minutes. [...] What if you want to switch over your web servers at a specific moment?

You don't. Configuration management is a wrong operation mode for a synchronous change. Still, you could order all your Puppet agents to run their scheduled operation now instead of leaving it waiting for its time.