Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"SRE" doesn't seem to mean anything useful any more

rachelbythebay.com

Post reply on HN
Print view View original

61–70 of 84 posts

Re: "SRE" doesn't seem to mean anything useful any more

#61

Post by kgeist »

joshstrange wrote: post #15

Honest question, what would you call a role that: * Is on call * Manages internal software (grafana, Prometheus, salt stack, etc) * is the first line of defense for issues in the field, works with support and the engineering team to handle problems * Manages a distributed fleet of servers (uses off the self and/or custom code to do so) * Builds internal tools/automations to improve the reliability of our platform and…

At our organization, a bunch of devs which were interested in improving/automating infrastructure split into a separate DevOps team. They first called themselves DevOps, then renamed themselves to SRE, then again to Production. Every time they changed their name they prepared presentations for the whole company explaining the name change and why the previous name wasn't correct after all. Everyone is still confused and always forgets what's their current official name.

Other than that, there's also:

- a separate IT department which manages networks, telephony and access

- the ones who push people to update software/OS is InfoSec, a separate team

- ordinary developers can make changes to k8s/grafana/Prometheus configs for their services, if it's reviewed by Production as the owners (they're understaffed compared to the multiple dev teams)

- there's also the "Core" team which deals with the microservice platform on top of bare infrastructure (how microservices should communicate, etc.)

Often the lines are blurred and it's not always clear who you call if you have a problem.

Re: "SRE" doesn't seem to mean anything useful any more

#62

Post by teqsun »

mwcampbell wrote: post #50
mike_hearn wrote: post #42

Earlier quoted context omitted.

Moreover, they must be cargo cults. An SRE role was my first job at Google in 2006. It was a product of its time and place. Google could hire the sort of full stack programmer/sysadmin types that the article talks about because they had enough clout and desirability at that time that they could get very skilled programmers to give up programming and do sucky on-call work instead. Even then, just two years after IPO a…

Aren't there things we can do to simplify the software stack, so we're more likely to find candidates with all the required skills in one package? There are certainly fashions we can push back against, e.g. premature use of microservices, Kubernetes, and distributed systems in general.

I don't think so, because there are countless vendors for all different parts of the stack fighting for marketshare and doing their best to prevent their competitors from becoming the de-facto for anything.

Just think of how many different flavors of SQL there are being run. If we can't even all agree on that, I have doubts that we could unify on the rest.

Re: "SRE" doesn't seem to mean anything useful any more

#63

Post by mike_hearn »

mwcampbell wrote: post #50
mike_hearn wrote: post #42

Earlier quoted context omitted.

Moreover, they must be cargo cults. An SRE role was my first job at Google in 2006. It was a product of its time and place. Google could hire the sort of full stack programmer/sysadmin types that the article talks about because they had enough clout and desirability at that time that they could get very skilled programmers to give up programming and do sucky on-call work instead. Even then, just two years after IPO a…

Aren't there things we can do to simplify the software stack, so we're more likely to find candidates with all the required skills in one package? There are certainly fashions we can push back against, e.g. premature use of microservices, Kubernetes, and distributed systems in general.

Cloud definitely helps, there. People pay money for clouds because they find ways to hire lots of SREs :)

For simplifying, yes probably, but it's hard. For instance relational databases require quite a lot of skill to run, even if you pay for a managed cloud DB. The cloud operator will handle some work like backups but some other problems like schema migrations, sudden query plan changes etc can still trip you up in production. To simplify that away you need a fundamentally different approach, maybe something like FoundationDB and Permazen. But then of course you lose a lot of the features that make people want the RDBMS engines.

Re: "SRE" doesn't seem to mean anything useful any more

#64

Post by nunez »

Well, yeah, in a world where corporates don't want to pay for sysadmins that can actually code or giving their devs a pager, you get what you saw: sysadmins and the people that do Jenkins being renamed as the "devops" team in 2014, then the "SRE" team in 2016 or so, then, "platform teams" after the age of Kubernetes in 2018-ish.

There are so many companies that have SRE teams despite those teams not maintaining a website!

Re: "SRE" doesn't seem to mean anything useful any more

#65

Post by nunez »

lr4444lr wrote: post #41

Ops people are cost centers. They can display their wizardry in blog posts until they are blue in the face, but except for those few companies with an incredibly large moat whose main profit is just raw traffic or high uptime (and how many companies like that can maintain that moat indefinitely anyway), engineers not actually building or directly improving product will always be cost centers. It brings me no joy to s…

Sucks, but true. Which is extremely unfortunate. Bad software is expensive to maintain, bad ops can take down companies, and both can slow the sales cycle and threaten renewals.

Engineering as a whole should be treated as a profit center.

Re: "SRE" doesn't seem to mean anything useful any more

#66

Post by nunez »

joshstrange wrote: post #15

Honest question, what would you call a role that: * Is on call * Manages internal software (grafana, Prometheus, salt stack, etc) * is the first line of defense for issues in the field, works with support and the engineering team to handle problems * Manages a distributed fleet of servers (uses off the self and/or custom code to do so) * Builds internal tools/automations to improve the reliability of our platform and…

That's reliability engineering, IMO.

Re: "SRE" doesn't seem to mean anything useful any more

#67

Post by nunez »

dijit wrote: post #5
candiddevmike wrote: post #4

It still means mandatory, potentially grueling, on-call rotations.

If you can affect the quality of the on-call, then it's not so bad. If you can't, then that's your ammunition to remove the feature from being "blessed". This is the gift that the SRE book actually gave people. That, and error budgets. It loosened peoples idea of what ops was, before this it was 100% uptime of all services at all times, and ops people being responsible for things that they could not reasonably affect…

Do you know of any company that actually _does_ error budgets? Like, they did the statistics on their previous downtime, calculated them and wrote SLOs with them baked in?

Re: "SRE" doesn't seem to mean anything useful any more

#68

Post by zdw »

dijit wrote: post #2

People need operations staff, people don't like operations staff and keep trying to treat them like developers. But, operations staff do and have always developed software, just internal software for glue or orchestration, and they work differently to regular software developers in that their customers are usually themselves to meet an internal objective of reliability, stability or ease-of-use for developers. It's i…

The older term "Systems Programmer" (which still exists primarily in academia, I had this title a few years ago at a university) has always felt more accurate to me as the "high level operations person who has programming as a primary skillset" job that the original SRE job description seemed to be aimed at.

Re: "SRE" doesn't seem to mean anything useful any more

#69

Post by benfortuna »

pjmlp wrote: post #33
benfortuna wrote: post #17

Earlier quoted context omitted.

Actually devs focusing on Ops lead to development of the automation we use today for most Ops activities, so I would say that is a success.

We already had such automation tools under system administration umbrella. What do we think we were using Perl for?

Scripting is not in the same league as Puppet, Chef, Ansible, etc. Automation at scale required the development of tools like these.

Re: "SRE" doesn't seem to mean anything useful any more

#70

Post by pjmlp »

benfortuna wrote: post #69
pjmlp wrote: post #33

Earlier quoted context omitted.

We already had such automation tools under system administration umbrella. What do we think we were using Perl for?

Scripting is not in the same league as Puppet, Chef, Ansible, etc. Automation at scale required the development of tools like these.

Most of those predate DevOps phenomenon, and have roots in systems administration.
Post reply on HN