Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sad state of sysadmin in the age of containers (2015)

vitavonni.de

Post reply on HN
Print view View original

421–430 of 435 posts

Re: The sad state of sysadmin in the age of containers (2015)

#421

Post by mmt »

auslander wrote: post #394
awakeasleep wrote: post #4

I want a prediction market to offer bets on which big company is affected by the breach that starts the reckoning.

I bet $100 on Google. It has most user data.

Except.. isn't that based on the misconception that all of these "cloud" tools that "come from Google" are actually used there?

Google isn't using Kubernetes with Docker on AWS (or even GCP). I'm quite confident that they're running their own software on custom bare metal, with custom virtualization/container abstraction layers and custom management software.

Re: The sad state of sysadmin in the age of containers (2015)

#422

Post by mmt »

tonyarkles wrote: post #418
mmt wrote: post #346

Earlier quoted context omitted.

I'm not convinced there was ever a need for sysadmins, especially from the point of view of programmers. Programmers have always been able to do that work themselves. It's just that they don't want to. They think it's boring, or perhaps even beneath them. It's a bit like tax preparation. I don't think that's really changed all that much, even today, even if it's now programming against the AWS API, judging by the num…

> I'm not convinced there was ever a need for sysadmins, especially from the point of view of programmers. Programmers have always been able to do that work themselves. > It's just that they don't want to. They think it's boring, or perhaps even beneath them. It's a bit like tax preparation. I'll half agree here, and half not. I'm coming at this as a programmer who has historically done ops as well, currently doing m…

> currently doing more ops than dev

I'm not sure you're even half disagreeing with me here, especially as you just admitted you may have the same bias as I do :)

> All of these cases have happened in recent memory, and it's never been a matter of "this is boring and/or beneath me", it's been a matter of "HALP! I HAVE NO IDEA WTF IS HAPPENING"

These two cases strike me as actually one case, with the former being the cause and the latter being the effect. At least that's my contention: they had no idea wtf is happening in those lower layers because learning them was beneath them (no pun intended). Nothing was really stopping them from learning about heap size versus process/cgroup limits.

On the other hand, you do bring up an important point about (mis)behavior of network infrastructure. That's not something the average programmer would necessarily encounter or have access to, especially at scale, and therefore wouldn't be expected to know. Still, my point still stands: there's nothing stopping programmers from gaining that knowledge if/when it becomes necessary, should they want to.

Now, I'm not trying to play dumb. I mostly don't want to read too much between the lines. Are you, in essence, saying that your half-disagreement is that sysadmins are needed, not because (average) programmers don't want to learn all those layers, but, instead, are incapable of learning them?

Re: The sad state of sysadmin in the age of containers (2015)

#423

Post by JdeBP »

mmt wrote: post #420
JdeBP wrote: post #407

Earlier quoted context omitted.

See these discussions, too: * https://news.ycombinator.com/item?id=15060146 * https://news.ycombinator.com/item?id=15060146

Did you mean to paste two different discussions? Those are identical.

Gah! Yes. The other one is:

* https://news.ycombinator.com/item?id=15044438

Re: The sad state of sysadmin in the age of containers (2015)

#424

Post by tonyarkles »

mmt wrote: post #422
tonyarkles wrote: post #418

Earlier quoted context omitted.

> I'm not convinced there was ever a need for sysadmins, especially from the point of view of programmers. Programmers have always been able to do that work themselves. > It's just that they don't want to. They think it's boring, or perhaps even beneath them. It's a bit like tax preparation. I'll half agree here, and half not. I'm coming at this as a programmer who has historically done ops as well, currently doing m…

> currently doing more ops than dev I'm not sure you're even half disagreeing with me here, especially as you just admitted you may have the same bias as I do :) > All of these cases have happened in recent memory, and it's never been a matter of "this is boring and/or beneath me", it's been a matter of "HALP! I HAVE NO IDEA WTF IS HAPPENING" These two cases strike me as actually one case, with the former being the c…

Very very excellent points. I think that everyone on my team would be capable of learning them, but at least in this specific case it's not so much that they feel it's beneath them, but more of a fear. They talk about C as if it's something that only dark wizards understand, likely as a result of poorly taught early CS.

I think what I'm saying is that I feel like sysadmins are frequently needed for sake of expediency, especially when hiring younger developers. And maybe, due to that same bias, I actually mean "senior developers who understand abstractions several layers deep" :)

Re: The sad state of sysadmin in the age of containers (2015)

#425

Post by mmt »

tonyarkles wrote: post #424
mmt wrote: post #422

Earlier quoted context omitted.

> currently doing more ops than dev I'm not sure you're even half disagreeing with me here, especially as you just admitted you may have the same bias as I do :) > All of these cases have happened in recent memory, and it's never been a matter of "this is boring and/or beneath me", it's been a matter of "HALP! I HAVE NO IDEA WTF IS HAPPENING" These two cases strike me as actually one case, with the former being the c…

Very very excellent points. I think that everyone on my team would be capable of learning them, but at least in this specific case it's not so much that they feel it's beneath them, but more of a fear. They talk about C as if it's something that only dark wizards understand, likely as a result of poorly taught early CS. I think what I'm saying is that I feel like sysadmins are frequently needed for sake of expediency…

> I think what I'm saying is that I feel like sysadmins are frequently needed for sake of expediency, especially when hiring younger developers.

Thanks. That's a point I hadn't considered.

I'm not sure that expediency translates to a need, as such, but that situation is certainly different from the one I envisioned (where a sysadmin is merely a luxury or an optimization to the programmer-DIY-ops scenario).

> I actually mean "senior developers who understand abstractions several layers deep" :)

I'm not sure you do, since you admitted to really being at least part sysadmin, earlier :)

Re: The sad state of sysadmin in the age of containers (2015)

#426

Post by tonyarkles »

mmt wrote: post #425
tonyarkles wrote: post #424

Earlier quoted context omitted.

Very very excellent points. I think that everyone on my team would be capable of learning them, but at least in this specific case it's not so much that they feel it's beneath them, but more of a fear. They talk about C as if it's something that only dark wizards understand, likely as a result of poorly taught early CS. I think what I'm saying is that I feel like sysadmins are frequently needed for sake of expediency…

> I think what I'm saying is that I feel like sysadmins are frequently needed for sake of expediency, especially when hiring younger developers. Thanks. That's a point I hadn't considered. I'm not sure that expediency translates to a need , as such, but that situation is certainly different from the one I envisioned (where a sysadmin is merely a luxury or an optimization to the programmer-DIY-ops scenario). > I actua…

> I'm not sure you do, since you admitted to really being at least part sysadmin, earlier :)

Hah!

I wish I could remember which talk it was. Bryan Cantrill had a good line about DevOps in one of his talks (I think it was this one: https://www.youtube.com/watch?v=30jNsCVLpAE). The gist was "you can say you're DevOps, but when the shit hits the fan, you're either going to be Dev or Ops. If you're a Dev, you're going to want to debug the problem before rebooting the failed machine. If you're Ops, you're going to want to reboot the machine as fast as you can to get it back up." Through that lens, I'm definitely pretty far over on the Dev spectrum; when something goes catastrophically wrong and someone reboots a box to "solve the problem", my first reaction is "YOU FUCKER YOU BURNED THE CORPSE"

It's such a tricky thing all around. Looking back at what I wrote earlier, I also realize that in some ways I'm facilitating the ignorance. I've got a really nice Consul and Nomad setup for the team, so they can pretty much just toss .wars and Docker containers at the cluster and they'll automatically get scheduled somewhere with spare capacity. The load balancer, the database cluster, all of the service discovery and job scheduling stuff... they've never had to get in and set any of that up. Maybe it's time to do more mentoring...

Anyway, thanks for getting me thinking. I've been a little bit grouchy lately about all of the recent experiences of people not knowing how the stuff they build actually runs. You've been a great mirror for some self-reflection :)

Edit: also, bare metal rocks.

Re: The sad state of sysadmin in the age of containers (2015)

#427

Post by peterbraden »

spookthesunset wrote: post #314
w4tson wrote: post #299

Earlier quoted context omitted.

I’ve wasted enough time tinkering with gulp, grunt and webpack to sympathize

> I’ve wasted enough time tinkering with gulp, grunt and webpack to sympathize And yet, those tools fill a need that would be very hard to replicate with the toolchains that came before it. Good luck doing half of what gulp / webpack do from, say, a Makefile.

The value of gulp / webpack et al are the plugins.

I've had success in several JS projects using make in combination with the shell version of a lot of these plugins and npm scripts.

There is value in both approaches.

Re: The sad state of sysadmin in the age of containers (2015)

#428

Post by seabrookmx »

steveklabnik wrote: post #411
jchw wrote: post #409

Earlier quoted context omitted.

Ah yeah, without CGO_ENABLED=0, you'll get a very cryptic error when the ELF binfmt can't find the linker binary... Never tried it with Rust, but I look to using Rust in the future, so I guess I better find out what the flags are for Rust. Sidenote: It's often useful to have ca certs and timezone info. At that point it's probably not a bad idea to just use Alpine and apk add those things.

By default, Rust compiles all Rust code statically, but the standard library depends on a libc. If you want to use MUSL, you can. If you bind to C libraries, you may need to configure it or not, it depends on how the wrapper is written.

Thanks for the clarification!

There was a recent HN article that did a step-by-step using rust+musl on Alpine. The "why" makes more sense now.

Re: The sad state of sysadmin in the age of containers (2015)

#429

Post by dagss »

snovv_crash wrote: post #130
dagss wrote: post #24

Earlier quoted context omitted.

I wish there was something like an updated Make, a tool that works for everything but updated to 2018. For instance Make works based on timestamps and therefore works very poorly together with git. Switch to another branch and you can get weird effects based on what files were updated and not and often trigger needless rebuilds. And everyone uses git these days. GNU Make, just using hashes instead of timestamps, woul…

CMake?

CMake generates a standard Makefile for dependency change tracking doesn't it? Just timestamp-based on hash-based?

Re: The sad state of sysadmin in the age of containers (2015)

#430

Post by dagss »

repolfx wrote: post #36
dagss wrote: post #24

Earlier quoted context omitted.

I wish there was something like an updated Make, a tool that works for everything but updated to 2018. For instance Make works based on timestamps and therefore works very poorly together with git. Switch to another branch and you can get weird effects based on what files were updated and not and often trigger needless rebuilds. And everyone uses git these days. GNU Make, just using hashes instead of timestamps, woul…

That'd require hashing every file on every run to figure out what changed. Not a good idea. There are modern tools that work for many things. Gradle supports native compilation these days as well as any JVM language. Bazel supports compiling many kinds of languages, and there's still scons.

I've tried scons, waf, CMake, etc, always ended up hating them and return to standard Makefiles. SCons is the worst I've tried ever.

As to the cost of hashing: "git status" runs on 0.009s and requires a full diff of my working directory?..

Post reply on HN