Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sad state of sysadmin in the age of containers (2015)

vitavonni.de

Post reply on HN
Print view View original

121–130 of 435 posts

Re: The sad state of sysadmin in the age of containers (2015)

#121

Post by jchw »

gaadd33 wrote: post #58
jchw wrote: post #25

Hadoop is a rather extreme example... It's bad, but not everything nowadays is. Many newer pieces of software install entirely from source with one command. Also, this is not at all endemic of containers, there's simply zero connection. Dockerfiles tend to be very simple and easy to reason about. The most complex application I have is around 50 lines of Dockerfile, and that's mostly just made more complicated to arra…

Does that Dockerfile build a container that only has that app in and it's required dependencies? Almost all of the ones I've seen given as examples seem to have an entire copy of the OS in.

A container is basically a glorified chroot, so there is a few things not strictly needed, but I typically use Alpine as a base system, which has a shell, some core utils, and musl libc in just ~5 MB. Since it's on its own layer, it gets deduplicated both in build and at runtime with other Alpine containers (and many Dockerhub images have an Alpine option.)

That being said, since Go binaries have no inherent dependencies, I have indeed made Docker images containing exactly one file: the Go binary. These containers are basically the same as fat binaries, with the benefits of Docker scheduling and networking at runtime.

Re: The sad state of sysadmin in the age of containers (2015)

#122

Post by notyourday »

kazen44 wrote: post #75
untog wrote: post #69

Earlier quoted context omitted.

> But I just don't understand why we have to have 47 half-built over-complicated build systems > Everyone repeat after me. I don't mean to pick on you specifically here because this attitude comes up a lot. In short, a lot of people are doing a thing, a thing that you aren't familiar with, and your gut reaction is to say "everyone: stop doing that, and do what I say!". Wait a second and consider that maybe there is a…

> Wait a second and consider that maybe there is a reason why this incredibly large number of developers are using these tools. That perhaps they evaluated various different options and decided that what they are using is more suitable than make. Maybe you could find out. Here we have another fundemental "problem" between dev and ops. The inherent friction because of different areas of concern. Dev's want to build fa…

Put devs on 24x7 escalation above your level-1 ("Is it actually broken?") NOC. Developer attitude would change immediately.

Re: The sad state of sysadmin in the age of containers (2015)

#123

Post by cookiecaper »

agentultra wrote: post #78
loteck wrote: post #23

The clearest explanation of why this happens is at the end: Before, admins would try hard to prevent security holes, now they call themselves “devops” and happily introduce them to the network themselves! 1) The merging of devs into the sysadmin role was a product of: the work of sysadmins (particularly systems change control and security compliance) not being valued in our culture. 2) Devs delighted to be free of th…

I recall the idea of "devops" from this book: https://landing.google.com/sre/book.html The stated goal of putting both systems administrators and software engineers on the same team is to reduce friction and increase communication. One of the worst, productivity-killing situations you can find yourself in when developing network software and services is caused by the traditional "old school" mentality of separating t…

The thing is that any separation in the roles in ineffective. Things shift around some if you embed an ops guy into the dev team directly, but it doesn't resolve the core problem. This applies to DBAs as well as ops or any other software-side segmentation as well.

The core problem is that there are "ops guys" and "dev guys". That creates conflicting incentives, even within the same team. It creates tension and a dynamic centered around bandying work around so that it's "the other guy's problem" in some situations, and hoarding logic onto the one segment so that there isn't an "obstruction" in getting things done in others. Moving the "segmented" guys directly into your team just makes these politics closer to the heart, which is not always an improvement.

Teams should be comprised of whole-platform "generalists" (in quotes because they really should be good at stuff, whereas "generalist" implies they aren't; here I just mean a competent non-specialist), where any single individual would be comfortable/capable performing any particular task that may come up. Of course, each member will have preferences and habits, little "skews", but it is important that these skews are controlled and used for mutual education, and not allowed to "flandersize" someone from "the guy who knows SQL better than the rest of us" to "full-fledged DBA who hasn't committed any C# for 3 years".

The right axis for separation is hardware v. software. If it's software-related, your dudes should be equally yoked, such that any SQL ticket would be assigned to any member of the team, or any "devops"/sysadmin/deployment ticket assigned to any member of the team.

These systems, from the OS up, are all part of the same thing, and they're all tightly integrated. Making the workload of the individual people on the team also tightly integrated is the only way to make sure that incentives align properly and that the most effective technical decisions are made, instead of decisions motivated, consciously or not, by offloading blame or other political/effort/convenience considerations that cause the overall system to suffer.

If you get into a sticky situation that requires specialized help from someone who has lived and breathed MySQL Server night and day, well, that's what consultants are for. Consultants would also be useful for inspections/sign-offs. But your core team can't tolerate being segmented out by component/implementation detail.

> Containers are a smart solution to the build problem.

Linux "containers" are a variety of things. True OS "containers" don't exist on Linux, but there are some rudimentary approximations. A Docker image is essentially a zip file, and sure, zip file-ish things may work fine for uploading artifacts to systems. Dockerfiles are unequivocally terrible, however.

Re: The sad state of sysadmin in the age of containers (2015)

#124

Post by mst »

eropple wrote: post #19
megaman22 wrote: post #14

Earlier quoted context omitted.

But I just don't understand why we have to have 47 half-built over-complicated build systems or job runners or whatever the new fad term is for every language, when there's something that does what they all do, is battle-tested, and has been around for decades. Everyone repeat after me. Makefiles are not scary. I can write a shell script. Do I really need to learn grunt/gulp/webpack/npm/rake/fake/maven/gradle/ant and…

Makefiles aren't scary. But they're also not particularly good. I use Rake (or Gulp, or whatever) because then I can use Ruby (or JavaScript, or whatever). Shell plumbing is fine for informal and small-scale stuff, and I make my code conform if somebody down the line (who may be me) wants to get out their duct tape, but the world is more complex than what /bin/sh can see. Shell is the lowest common denominator. Expec…

> `JSON.parse` and a working subscript operator

https://github.com/ingydotnet/json-bash

    source json.bash
    
    json='{"name":"Jason","friends":["Jimmy","Joe"]}'
    JSON.load "$json"
    joe=$(JSON.get /friends/1)
    JSON.put /friends/2 Jeff
    new_json=$(JSON.dump)
It's still not as trivial as javascript, but "tooth-pullingly difficult" is a little unfair.

Re: The sad state of sysadmin in the age of containers (2015)

#125

Post by colanderman »

untog wrote: post #83
kazen44 wrote: post #75

Earlier quoted context omitted.

> Wait a second and consider that maybe there is a reason why this incredibly large number of developers are using these tools. That perhaps they evaluated various different options and decided that what they are using is more suitable than make. Maybe you could find out. Here we have another fundemental "problem" between dev and ops. The inherent friction because of different areas of concern. Dev's want to build fa…

Not really? A lot of build tools are chosen specifically to make the build process more reliable and understandable. For example, build tools like Maven handle dependency management, which is hugely beneficial in ensuring your builds are consistent and work the same in different environments. Makefiles are shell scripts.

Makefiles are definitely not shell scripts. Individual recipe lines are executed using Bash by default, but you can change this. (Heck you probably could even get away with Python if you really wanted.)

The rest of Make (which is 95% of what's in a Makefile) is its own language, which is actually not too bad for what it's intended for (munging around filenames), and has the flavor of a functional language.

Re: The sad state of sysadmin in the age of containers (2015)

#126

Post by mancerayder »

gigatexal wrote: post #7

Ugh make files. Kill me. Give me a bash script or give me death.

It's trivial to call bash syntax from within a Makefile. The Makefile just breaks these things up into rules which define steps and dependencies for these rules to occur.

Re: The sad state of sysadmin in the age of containers (2015)

#127

Post by takeda »

notyourday wrote: post #105
agentultra wrote: post #78

Earlier quoted context omitted.

I recall the idea of "devops" from this book: https://landing.google.com/sre/book.html The stated goal of putting both systems administrators and software engineers on the same team is to reduce friction and increase communication. One of the worst, productivity-killing situations you can find yourself in when developing network software and services is caused by the traditional "old school" mentality of separating t…

> Don't build your containers from public, un-trusted images! Build your own images. Run your own, protected, registry. You still have all of the compliance and validation necessary and you don't end up debugging failed builds because one machine out of a thousand is running on some minor shared library version not supported by your software. You have just lost all the speed to production advantages of containers.

If that's the reason, then this is broken from the start.

BTW the more I learn and use nix the more I see that this is the proper solution to what docker is currently marketed to fix.

Re: The sad state of sysadmin in the age of containers (2015)

#128

Post by bluGill »

notyourday wrote: post #105
agentultra wrote: post #78

Earlier quoted context omitted.

I recall the idea of "devops" from this book: https://landing.google.com/sre/book.html The stated goal of putting both systems administrators and software engineers on the same team is to reduce friction and increase communication. One of the worst, productivity-killing situations you can find yourself in when developing network software and services is caused by the traditional "old school" mentality of separating t…

> Don't build your containers from public, un-trusted images! Build your own images. Run your own, protected, registry. You still have all of the compliance and validation necessary and you don't end up debugging failed builds because one machine out of a thousand is running on some minor shared library version not supported by your software. You have just lost all the speed to production advantages of containers.

No, you still get that. You are mistaking initial speed to production (longer with containers) for amortized speed to production when you scale.

Re: The sad state of sysadmin in the age of containers (2015)

#129

Post by pjz »

lostcolony wrote: post #96
lbenes wrote: post #80

Earlier quoted context omitted.

There are nearly 10 million lines of code in libreoffice, and yet I can and have built it from source just by typing: 1. $ git clone git://anongit.freedesktop.org/libreoffice/core 2. $ apt-get build-dep libreoffice 3. $ ./autogen.sh && make Just because something has a large code base doesn't mean we shouldn't be able to build it from source ourselves.

Did you read all those lines yourself? Did you even confirm checksums matched before running them? I think that's the parent's point. You can build from source, but how do you trust the source? Is it any more egregious to trust a prebuilt binary from a specific website than it is the raw source? If you can't trust the binary being hosted by the author/caretaker, can you really trust the source being hosted or maintai…

I don't think his point is so much about the source as it is about updating N containers. For instance, say there's a known libssl bug. Can you tell how many of your containers are running that version of libssl? And how do they get updated?

Re: The sad state of sysadmin in the age of containers (2015)

#130

Post by snovv_crash »

dagss wrote: post #24
megaman22 wrote: post #14

Earlier quoted context omitted.

But I just don't understand why we have to have 47 half-built over-complicated build systems or job runners or whatever the new fad term is for every language, when there's something that does what they all do, is battle-tested, and has been around for decades. Everyone repeat after me. Makefiles are not scary. I can write a shell script. Do I really need to learn grunt/gulp/webpack/npm/rake/fake/maven/gradle/ant and…

I wish there was something like an updated Make, a tool that works for everything but updated to 2018. For instance Make works based on timestamps and therefore works very poorly together with git. Switch to another branch and you can get weird effects based on what files were updated and not and often trigger needless rebuilds. And everyone uses git these days. GNU Make, just using hashes instead of timestamps, woul…

CMake?
Post reply on HN