Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Eulogy for DevOps

matduggan.com

Post reply on HN
Print view View original

141–150 of 175 posts

Re: A Eulogy for DevOps

#141

Post by sgarland »

michaelmior wrote: post #125
hparadiz wrote: post #119

Earlier quoted context omitted.

I honestly don't get this thread and the complaining. Yes, I was one of those empowered engineers at an org that did ALL of the DevOps. But do ya'll seriously think you don't sometimes need people who do ONLY DevOps? Cause I can tell you there are absolutely orgs that deploy dozens, if not hundreds of servers sometimes by region and in those cases you absolutely do need full time DevOps engineers to tune, configure,…

> in those cases you absolutely do need full time DevOps engineers to tune, configure, and deploy these fleets At that point, it sounds like you're just doing ops work.

Correct, except generally they aren’t even good at ops. They tend to know a highly specific slice, or how to operate an abstraction. But troubleshooting Linux? Good luck.

Re: A Eulogy for DevOps

#142

Post by sgarland »

betaby wrote: post #61
milkglass wrote: post #48

Earlier quoted context omitted.

Where's all the good operations people at nowadays? Have worked with numerous cloud native engineers that do not have good foundational knowledge.

> cloud native engineers That's the problem. Yes, I'm the old man yelling at cloud, but indeed 'kids this days' don't know Linux, they know how to provision things from Terraform. In fact it feels like logging in via ssh and checking process with with `strace` is a lost art. Checking PCAP? That's a black magic!

It’s the logical outcome for most when you don’t grow up being told to RTFM, you’ve never had to recompile a kernel to get an expansion card to work, and you’ve never touched hardware.

Re: A Eulogy for DevOps

#143

Post by sgarland »

sadops wrote: post #68
betaby wrote: post #61

Earlier quoted context omitted.

> cloud native engineers That's the problem. Yes, I'm the old man yelling at cloud, but indeed 'kids this days' don't know Linux, they know how to provision things from Terraform. In fact it feels like logging in via ssh and checking process with with `strace` is a lost art. Checking PCAP? That's a black magic!

This is the elephant in the room: all "new" technology isn't fundamentally new. Terraform is just curl with state management for lots of different websites, Andible is just a YAML to Bash converter, and so on. If you have good fundamentals, not only are these things easy, they're also incredibly frustrating, because you can easily see their limitations. You can always tell how experienced someone is by how well they…

> There's no substitute for skill and experience, despite what the modern tech discourse says. "Anyone can program!"

I’m told this is gatekeeping. Yes, and…? I’ve always found it amusing that I see little to no anger being directed at kernel devs for their gatekeeping. Almost as if deep down, people know that they shouldn’t be fucking with the thing that runs the world unless they are actually good at it. Or maybe they’re just afraid of Torvalds.

Re: A Eulogy for DevOps

#144

Post by osigurdson »

zer00eyz wrote: post #113
osigurdson wrote: post #42

Earlier quoted context omitted.

>> "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would make the problem go away I'm not familiar with the reference but logs, events and metrics seem pretty useful. Port forward, shell into containers, view logs, etc. I don't see what is so bad about it.

> shell into containers The problem here is that once your in and need tooling what do you do? Your likely going to have to re-build the whole container with that tooling on board (if its even possible)... Containers aren't problematic in themselves. There are reasons to use them. Ruby, php, JS python (fucking venvs) have this habit of contaminating a system. Containers are a great way of... containing that shit. But…

Even with a single file binary, there are still reasons to put it in a container. You could do some similar things (e.g. resource limits) messing with cgroups manually, but why? So much tooling has been built up around containers that not using them often means re-inventing things that are already done.

Of course, if you have one file (or many) that you run on one server and downtime is ok during updates then fine to just do everything manually. You could even keep the code on that server as well. ssh in, edit compile and run your production app directly from the out directory. Basically run production on your dev machine. It sounds pretty silly / "non professional", but there are times where ultra low cycle times outweigh other things (like when you have one developer and desperately trying to win your first customer - you know "do things that don't scale"). The main thing is, you need to know the trade-offs you are making.

Re: A Eulogy for DevOps

#145

Post by sgarland »

altdataseller wrote: post #31
giantg2 wrote: post #7

Earlier quoted context omitted.

But who needs to know what happens behind the facade of the services? It does everything for us! I miss the days of having an expert sys admin and DBA to help with tough issues. Now we're all on our own with no training. Most of the good people got axed or thrown into other roles. Becoming good at something only to have it thrown away and undervalued pisses me off.

What sort of skills does a sysadmin and DBA have that most engineers lack? Most engineers should know the basics of bash scripting, iptables, Unix logging, cron, systemd, etc. as well as SQL, debugging slow queries, optimizing them, backing up dbs, etc

> basics of bash scripting…

Bash alone has a million footguns. If you know to use shellcheck you can probably survive those, and if you read the bash manual in its entirety you’ll almost certainly be well on your way to greatness, but getting devs to read docs – let alone boring, Web1.0 docs, is stretch.

> basics of SQL…

Have you seen the knobs Postgres and MySQL have to turn? Do you know what they all do, when you should turn them, and by how much?

The documentation for both of these is enormous and highly detailed, yet in no way covers everything that can and will go wrong. You only find those things out by using it day in and day out.

As to SQL itself, “I can do inner joins” is about as complex as I’ve seen most devs do. Hell, mention a semijoin and you’ll get blank stares.

I think in general, you’re grossly overestimating the average developer’s knowledge breadth and desire to learn these things. They simply do not matter for most, because tooling exists such that most of the time, they don’t need to know. I consider myself decent at Linux and RDBMS, but that’s mostly because I genuinely enjoy both, play with them in my off-time, and have been running Linux in some form or fashion for the last 20+ years. Also, of course, my work specializations (SRE —> DBRE) have helped.

Re: A Eulogy for DevOps

#146

Post by sgarland »

wild_egg wrote: post #40
altdataseller wrote: post #31

Earlier quoted context omitted.

What sort of skills does a sysadmin and DBA have that most engineers lack? Most engineers should know the basics of bash scripting, iptables, Unix logging, cron, systemd, etc. as well as SQL, debugging slow queries, optimizing them, backing up dbs, etc

_should_ know is different than _do_ know. Most engineers have little to no understanding of those systems topics and the only DB knowledge they've picked up is what leaks through their ORM A person who competently knows all those topics _and_ how to write application code is worth their weight in gold

> A person who competently knows all those topics _and_ how to write application code is worth their weight in gold

IFF the company consistently values those skills. IME, they’ll say upon interviewing or hiring that those aren’t necessary for their workload, until they suddenly are, where you’re lauded as a hero. Predictably, the memory of hero status fades when promo season comes around, because those “aren’t core skillsets,” or something similar.

Re: A Eulogy for DevOps

#147

Post by turtleyacht »

osigurdson wrote: post #42
zer00eyz wrote: post #5

> Money was (effectively) free so it was better to increase speed regardless of monthly bills. Jesus this. No one knows where the money goes. If you can't tell me cost per customer, per user then your business is missing key metrics. > ... "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would m…

>> "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would make the problem go away I'm not familiar with the reference but logs, events and metrics seem pretty useful. Port forward, shell into containers, view logs, etc. I don't see what is so bad about it.

In the far future, technology is so advanced that "rituals" are invented to "appease the machine spirits." No one knows how it works anymore, and everything must support an immense galactic, perpetual war effort.

Re: A Eulogy for DevOps

#148

Post by osigurdson »

zer00eyz wrote: post #113
osigurdson wrote: post #42

Earlier quoted context omitted.

>> "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would make the problem go away I'm not familiar with the reference but logs, events and metrics seem pretty useful. Port forward, shell into containers, view logs, etc. I don't see what is so bad about it.

> shell into containers The problem here is that once your in and need tooling what do you do? Your likely going to have to re-build the whole container with that tooling on board (if its even possible)... Containers aren't problematic in themselves. There are reasons to use them. Ruby, php, JS python (fucking venvs) have this habit of contaminating a system. Containers are a great way of... containing that shit. But…

If you need a container to be mutable for a while, you can. Mount an NFS volume and run from there.

You can even checkout your code, edit in vim and re-deploy. Crazy talk for production of course, but if you need it, it is possible. Basically it is not that far off from running a binary on a server if you need it to be.

Re: A Eulogy for DevOps

#150

Post by 28304283409234 »

osigurdson wrote: post #37

>> abandon technology like Kubernetes I think a lot of Kubernetes hate is misplaced. It is a great piece of software engineering, well supported and runs everywhere. You certainly don't always need it but don't create a bunch of random bash scripts running all of the place instead of learning how to use it.

Many things are 'great pieces of [software] engineering'. That does not mean they are a fit solution to any problem. Many times a script or simple ansible playbook, or docker-compose is just a better fit.
Post reply on HN