Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postmortem of my 9 year journey at Google

tinystruggles.com

Post reply on HN
Print view View original

121–130 of 255 posts

Re: Postmortem of my 9 year journey at Google

#121

Post by hn_throwaway_99 »

advisory5739f2 wrote: post #109
rr808 wrote: post #29

I've done the path from SWE to SRE and back to SWE. I was always happy to do production support and diagnose and fix production problems, so I naturally moved to SRE which is always looking for people. It was a real mistake, SRE is hugely stressful and really unrewarding compared to SWE. Yes you learn some skills and get some occasional glory, but year after year of fighting fires really didn't build any long lasting…

SRE is an anti pattern that Google is unwilling to admit and is selling books on. Just like there should not be QA, release engineering, continuing engineering or DBA as separate departments/job titles, because these critical parts of software development should not be considered optional and thrown over the wall to take care of by someone with no stake in developing the product.

I've been one the other side of this (i.e. companies that have no SREs or QA, or in one case a company that had QA and got rid of it) and it has always been an unmitigated disaster.

The root cause of this disaster is that, when writing software, interruptions are the death of productivity. Having a software engineer wear too many different hats at one time, especially when some of those hats are largely real-time interrupt driven, can absolutely kill productivity.

To emphasize, I'm not at all in favor of "throwing things over the wall". Software engineers are responsible, for example, for making software that is easy to test and has good observability in place for when production problems show up. But just because you listed a bunch of things that are "critical for software development" doesn't mean that one person or role should be responsible for all of these things.

At the very least, e.g. for smaller teams I recommend that there is a rotating role so devs working on feature development aren't constantly interrupted by production issues, and instead each dev just gets a week-long stint where all they're expected to do is work on support issues and production tooling improvements.

Re: Postmortem of my 9 year journey at Google

#122

Post by lexlash »

bzhang255 wrote: post #45
redrix wrote: post #20

Earlier quoted context omitted.

Isn’t this the whole point of working; particularly at a FAANG? > It’s a shallow post-mortem I respectfully disagree. It’s an 8 minute read. Sure, it’s mostly in dot-point form, but personally I’d rather that than some massive 80,000 word blog post that I’m going to drop 1/8 of the way through. Since when does a personal blog post need to be a well constructed and lengthy document?

I think it's shallow and not because it's short. To me, it just sounds very typical: "I joined Google back when it was fun. Now it's more bureaucratic and less fun. But I made a ton of money on the stock." I think there have been countless blog posts from Ex-Googlers like this. It's fairly shallow. And it is worth noting that a lot of the bullet point lists do start with "I made a ton of money" in as many words, whic…

The most interesting thing is the timeline at the end, which shows what they were successful at and promoted for (management-type roles) and what happened when they tried to transition from SRE management to SWE IC (they fell back into management).

I don't see that reflected in the rest of their postmortem learning - other than them being dissatisfied with doing what they were good at / promoted for - so that kind of helps me ignore the rest of the postmortem. :)

Re: Postmortem of my 9 year journey at Google

#123

Post by astral303 »

dexwiz wrote: post #116
advisory5739f2 wrote: post #109

Earlier quoted context omitted.

SRE is an anti pattern that Google is unwilling to admit and is selling books on. Just like there should not be QA, release engineering, continuing engineering or DBA as separate departments/job titles, because these critical parts of software development should not be considered optional and thrown over the wall to take care of by someone with no stake in developing the product.

This is such a crazy take for me. Any profession that matures eventually specializes. I wouldn’t expect the same person to pour my foundation, install the plumbing, and wire the building. Yet in an ever expanding field we expect someone to be able to do it all. Also saying people who don’t code have no stake so egocentric.

Pouring foundation, installing plumbing and wiring the building is specialized by the physical necessity of these activities, which cannot be repeated without mistake at a great cost. That justifies specialization. Unlike building a bridge, compiling software is essentially free. QA, release engineering and database design can and should be repeated and iterated on by software engineers, because it is a necessary part of the development and removing it from the expected work distorts incentives.

Re: Postmortem of my 9 year journey at Google

#124

Post by tw04 »

ethbr1 wrote: post #40
darkwater wrote: post #35

Earlier quoted context omitted.

Indeee. I think because for younger generation is physically impossible to have experienced it, while for the older generations it's complicated to get into a disruptive startup. Obviously people could read about the past, but sometimes that's asking too much, they are busy creating "the future".

Also, you know, there probably aren't that many posters on HN who worked at AT&T in the 60s...

If you can’t be bothered to read up on why previously great companies fell from grace, you’re kind of begging to repeat their mistakes…

Re: Postmortem of my 9 year journey at Google

#125

Post by cperciva »

lexlash wrote: post #119
cperciva wrote: post #115

Earlier quoted context omitted.

Ok, so at Google L3-L5 are junior developers, L6/L7 are developers, and L8+ is senior developers?

I'm not sure what you mean by "junior" here. L3 is early career, L4 is mid-career, L5 is senior. You can hit L5 on the strength of pure technical contributions regardless of business/org needs, usually. L6+ is staff, and tends to involve a very different skillset. (If you're not looking to lead a team, you're probably not going to have the kind of impact that gets you to L6, let alone L7 or higher at Google.) This is…

My definitions are basically: Junior developers need supervision because left to their own devices they'll screw things up horribly; normal developers can produce good code independently; and Senior developers are able to catch the mistakes the Junior developers are making and set them on the right path.

Re: Postmortem of my 9 year journey at Google

#126

Post by lexlash »

cperciva wrote: post #125
lexlash wrote: post #119

Earlier quoted context omitted.

I'm not sure what you mean by "junior" here. L3 is early career, L4 is mid-career, L5 is senior. You can hit L5 on the strength of pure technical contributions regardless of business/org needs, usually. L6+ is staff, and tends to involve a very different skillset. (If you're not looking to lead a team, you're probably not going to have the kind of impact that gets you to L6, let alone L7 or higher at Google.) This is…

My definitions are basically: Junior developers need supervision because left to their own devices they'll screw things up horribly; normal developers can produce good code independently; and Senior developers are able to catch the mistakes the Junior developers are making and set them on the right path.

I see; that's L3, L4, and L5 progression in a nutshell at Google - although leaving L3s alone doesn't _guarantee_ something will go wrong, it was more that there was no way for them to figure out optimal solutions without help thanks to the sheer complexity of Google.

I'd say the same held true at Amazon but I was in groups which were, at the time, at the periphery of the company's engineering efforts - we didn't have any associated principals to talk to, and maybe one SDE3/L6 to 10 SDE2/L5s mixed with SDE1/L4s.

Re: Postmortem of my 9 year journey at Google

#127

Post by Rebelgecko »

danpalmer wrote: post #24

There are a bunch of comments saying that Google is just like any other big tech company and that the exciting engineering bit has gone. My experience is only from the last 2.5 years, but I've got a slightly different take. Engineering from >10 years ago seems like it was a wild west. Some truly stunning pieces of technology, strung together with duct tape. Everything had its own configuration language, workflow engi…

On the flip side, clueless execs are killing tools like Code Search because they don't understand the value. They're willing to layoff off a team to save money even if it reduces the productivity of 20,000 employees by 1%

Re: Postmortem of my 9 year journey at Google

#128

Post by advisory5739f2 »

hn_throwaway_99 wrote: post #121
advisory5739f2 wrote: post #109

Earlier quoted context omitted.

SRE is an anti pattern that Google is unwilling to admit and is selling books on. Just like there should not be QA, release engineering, continuing engineering or DBA as separate departments/job titles, because these critical parts of software development should not be considered optional and thrown over the wall to take care of by someone with no stake in developing the product.

I've been one the other side of this (i.e. companies that have no SREs or QA, or in one case a company that had QA and got rid of it) and it has always been an unmitigated disaster. The root cause of this disaster is that, when writing software, interruptions are the death of productivity. Having a software engineer wear too many different hats at one time, especially when some of those hats are largely real-time int…

I agree very much with you that interruptions are death of productivity. Your suggestions for weekly rotations are great.

However, I argue that if the engineers are interrupted by QA issues, they will be motivated to find ways to not have those QA issues. In absence of that, we end up with the familiar “feature complete, let QA find bugs” situation.

Re: Postmortem of my 9 year journey at Google

#129

Post by sangnoir »

advisory5739f2 wrote: post #128
hn_throwaway_99 wrote: post #121

Earlier quoted context omitted.

I've been one the other side of this (i.e. companies that have no SREs or QA, or in one case a company that had QA and got rid of it) and it has always been an unmitigated disaster. The root cause of this disaster is that, when writing software, interruptions are the death of productivity. Having a software engineer wear too many different hats at one time, especially when some of those hats are largely real-time int…

I agree very much with you that interruptions are death of productivity. Your suggestions for weekly rotations are great. However, I argue that if the engineers are interrupted by QA issues, they will be motivated to find ways to not have those QA issues. In absence of that, we end up with the familiar “feature complete, let QA find bugs” situation.

> However, I argue that if the engineers are interrupted by QA issues, they will be motivated to find ways to not have those QA issues.

There are institutional limitations that engineers cannot overcome, no matter how zealous or motivated. Moreover, companies also ought to remember that engineers can "find ways to not have those QA issues" by seeking employment elsewhere!

Re: Postmortem of my 9 year journey at Google

#130

Post by jorams »

skybrian wrote: post #103
justinclift wrote: post #88

Earlier quoted context omitted.

> I think caused public opinion to shift is when people started realizing how truly massive they had gotten. I'm personally more of the opinion that Google has caused enough serious issues for enough people - with famously no way to get the issues resolved - that they themselves seeded or caused the negative public opinion. Combine that with Google behaviour of clearly doing things in their own interest even when not…

I don't think people understand what manifest v3 is trying to solve. It's a good example of how a worthwhile effort to make the Internet less terrible can be misinterpreted.

Manifest v3 is an effort to make a browser slightly better at its core feature, that it is importantly already fine at, at the cost of making it worse at serving the user. The internet is increasingly user-hostile, and manifest v3 makes it harder to fight back.
Post reply on HN