Live data from Hacker News

Software Engineering at Google

arxiv.org

101–110 of 161 posts

Re: Software Engineering at Google

#101
post #84
post #78

Earlier quoted context omitted.

Funnily when I got to visit the Google office in London people just left their trash on the floor of the cafeteria for the cleaners..

I think his comment was mostly figurative.

Except no, I saw stuff like that too when I was there along with childish crap like throwing gum in the urinals and googlers acting like entitled jerks towards the cleaning staff.

I met some great people at Google, but I'd be remiss not to mention I also met too many #IAmGoogle sorts (who were all dudes BTW).

Re: Software Engineering at Google

#102

So much snark and negativity on this thread, when the paper is just a factual description of a pretty impressive feat of software engineering (still way better than most companies I have seen the inside of).

I really like the first part of the paper and it describes a lot of their best practices, but the last 1/3 or so of the paper is more about Google's culture, no? IMO that opens them up for the snark here.

Google has some amazing perks and it's a wonderful overall employer, but I really wonder how much cash is left on the table by not addressing its shockingly low retention rate or at least explaining why it's not a problem.

There are a lot of articles estimating the cost of a departure is anywhere from 6-12 months of the lost employee's salary. Does that seem like something that should be ignored like they seemingly do? Why do shareholders even tolerate this?

Re: Software Engineering at Google

#103

Google is highly non-representative of businesses and technology businesses in particular. It has a near monopoly on the search business and has enormous amounts of money from a single source -- advertising. Google Fiscal Year 2015 Revenues: $74.54 Billion (source: Google) Advertising Revenue: $67.39 Billion (source: https://www.statista.com/statistics/266249/advertising-reven... ) Profits: $23.4 Billion Market Capit…

I agree. Many companies, including mine, run critical stuff on applications that are over a decade old that do just fine. Worst thing you can say about them is that some weren't designed well enough for easy extensions or integration with newer apps. They do their job, though. The backbone of ours is mainframe code running on mainframes and AS/400's with simple, terminal interfaces. Terminal stuff is ultra-fast & reliable but sometimes ugly. Some have GUI apps that basically hide the terminal details they interact with to be a bit easier to use. Those terminal apps, probably 20+ years old, still work and get periodically updated. New people learn them easily, too, since interface was well-designed for the time. Can't goof off on the thin clients either as there's no web browser or native apps. ;)

I've seen Google do some rewrites that make sense when one has the money for them. The shift from eventual to stronger consistency in their databases via F1 RDBMS was impressive. Worth some rewrites across the apps to use it to knock out major problems that could affect them once and for all. After developing Go, they also might want to rewrite performance-critical apps in, say, Python to it. There's definitely benefits on such rewrites. A lot of the other stuff I'm betting they could've done more long-term esp if using and extending FOSS solutions.

Re: Software Engineering at Google

#104
post #56

Interesting, compared to what's common in the automotive industry it doesn't even mention the terms "requirements", "specifications", "estimations", "project plan", "tracability", "UML", etc...

Right, if it mentioned those, search would still be the version we saw in 2007. None of the other products would exist yet because they would still be conducting user studies based on wire frame mock ups of workflows designed by committees of behavioral psychologists. That's a bit dramatic, but when your product development has a fast turnaround for fixes (git push vs 100 million dollar recall) and it won't kill peop…

A clear statement of what you're going to do, some constraints on the design a la Design-by-Contract, and languages/libraries that mitigate errors by design are so easy to do that small shops do them on a regular basis. Ada/SPARK, Eiffel, Ocaml, and Haskell are examples with steady business in industry with last three used on relatively-fast-moving projects. Add in static analysis, spec-based generation of tests, and/or fuzzers to get lots of reliability for free. Guess what? This method also scales excellently if the company has access to a huge pile of servers and engineers whose build system can automate all the checks with every submitted change.

Your idea that it has to be as ridiculous as process junkies is a strawman. A strawman that happens in a lot of places for sure but doesn't have to. Google can just take the few, easy-to-apply practices from high-integrity systems to get tons of benefit. It's the 80/20 rule for increasing assurance.

Re: Software Engineering at Google

#105
I read most of the paper. For the most part, it struck a nice tone as being mostly descriptive and not too promotional. However, the final paragraph in the conclusion section differs:

> "For those in other organizations who are advocating for the use of a particular practice that happens to be described in this paper, perhaps it will help to say “it’s good enough for Google”.

In my opinion, this style of writing doesn't fit nor belong.

I would leave that paragraph out. Instead, let's judge on the merits and applicability of an engineering practice based on thinking, reasoning, and experimentation.

That said, as I've read various comments about Google's processes, I'm struck by the cognitive dissonance. On one hand, I see bandwagoning; e.g. "monolithic source control is nuts; we don't do that; no one I know does that". There is also some appeal to authority; e.g. "well, Google is the best, they do X, so we should too." I'm glad to see different argumentation fallacies colliding here.

Re: Software Engineering at Google

#106

Earlier quoted context omitted.

Everything you posted is wrong, which makes me believe you know everything you posted is wrong. Nobody writes things that inaccurate by accident. For the benefit of YC, here are corrections: > No branches. If you want to make a temporary code branch, > you create a CL (Google's version of a pull request), but > never submit it. This means nobody else can collaborate on > it with you, and it must be manually updated t…

Thanks for the info! > Piper has branches I've worked on two teams so far, neither of which used the branches you described. They opted to either flag off features or keep long-running CLs. However, I'll try to learn more about Piper branches. I doubt my team will make large workflow shifts, but it would still be good to understand! > CLs are equivalent to Git commits I'd argue that CLs are not equivalent to Git comm…

I don't know which team you work on, so this advice might not be appropriate for your situation. Feel free to email me and I'll try to help you get things sorted. I won't make any attempt to link your corp username to these comments. Same for any other Googlers reading this chain.

  > I've worked on two teams so far, neither of which used the
  > branches you described. They opted to either flag off
  > features or keep long-running CLs.
  > However, I'll try to learn more about Piper branches. I
  > doubt my team will make large workflow shifts, but it
  > would still be good to understand!
Putting new features or changed behavior behind a flag is a good process. Encourage your teammates to avoid long-running CLs in favor of submitting flag-guarded code. You should treat every run of [g4 patch] as a warning that the CL is dragging on too long.

Dev branches require a change in workflow, and can be unhelpful if a team has bigger process issues (like yours sounds like). I recommend looking into them and trying out the codelab, but don't start advocating for them just yet.

Release branches are very important. If your team is not currently using them, that needs to be fixed ASAP. Look into Rapid[1], try it out for a small CLI utility. Advocate for all releases to be done via Rapid. Despite the name it can be slower than plain [g4 sync; blaze build] but the extra steps it runs are important and useful.

[1] https://landing.google.com/sre/book/chapters/release-enginee...

  > I'd argue that CLs are not equivalent to Git commits, the
  > equivalent would be a CL snapshot. Best practices in git
  > are to have frequent, small commits. CLs tend to be much
  > larger, and the review process means that having small
  > CLs would greatly slow workflow.
I'm a Go readability reviewer, so I review a lot of CLs written by people in other parts of the company. My firm belief is that CLs should be small and frequent. CLs start to get hard to review at around 300 lines of feature code. If you are regularly reviewing large CLs, push back and ask the authors to split them up. Often these CLs are trying to do too many things at once and you can find a good fracture point to split them into 2-3 parts.

If you are regularly writing large CLs, or long chains of diffbase'd CLs, that's a sign that your codebase may be poorly factored. Take a step back from the tactical level and look at what your CLs are touching. Is the UI's HTML formatting mixed into the business logic? Are you touching the same file over and over? Move things around, use includes, use data embeds. Replace long param lists with data structures. All this standard software engineering advice applies 2x when working with other devs.

  > Yes, but this doesn't help with development. When HEAD is
  > broken, a developer has to chose between developing on an
  > outdated codebase, or developing on a broken codebase.
If HEAD is broken, your first priority should be getting HEAD fixed. Whether that means fixing the code or rolling back to an earlier version, you should not accept a broken HEAD.

After it's fixed, look at why it broke. Why was a CL that broke things allowed to be submitted? Do you have proper presubmit tests? Consider adding your TAP project(s) to METADATA files so Piper will make sure tests pass and binaries build before allowing the submit.

If other teams' changes are breaking your code, add your TAP project(s) to their METADATA or help them improve their own test coverage.

  > After a brief correspondence with the team that manages
  > third-party libraries, I decided it would be easier to
  > implement the feature myself instead of following
  > whatever process was required to update the library.
Third-party code has special rules that might prevent you from doing something reasonable. Ask the team for help with the process. It's a natural developer instinct to write new code instead of trying to update shared dependencies. If you can fight that instinct and get the new dep version imported, it will improve not just your project but the projects of everyone who might use that dep in the future.

Re: Software Engineering at Google

#107
post #96

Not one word about what I consider the most toxic aspect of working at Google: blind allocation. Unless one absolutely does not care what one wishes to work on, joining Google is throwing your future into the Hogwarts hat of an ill-defined cabal of billionaires to pick your job at Google. Sometimes that works out, but most of the time you are allocated to whatever mission-critical project is currently leaking buttche…

Blind allocation?

When I joined Google I was given 4 teams in 2 different locations to choose. I had lunch with each of the teams, and chose the one that better suited my style.

To be fair, your experience at Google will depend a lot on what team you land in. Not everyone's given a choice, but now I know better: Talk to your recruiter and make them clear what you want.

Re: Software Engineering at Google

#108
post #96

Not one word about what I consider the most toxic aspect of working at Google: blind allocation. Unless one absolutely does not care what one wishes to work on, joining Google is throwing your future into the Hogwarts hat of an ill-defined cabal of billionaires to pick your job at Google. Sometimes that works out, but most of the time you are allocated to whatever mission-critical project is currently leaking buttche…

I didn't even apply until finding a team I was really excited about. I had coffee with a couple TPMs and managers, found the right group, applied, and have been really happy. The median tenure of the folks I work with is 5 years, and so far it's been great.

Re: Software Engineering at Google

#109

Earlier quoted context omitted.

Right, because data leaves no room for interpretation.

No need to be needlessly sarcastic. Data-driven means that you collect various metrics on dev workflow, what slows productivity, or on the product side (user patterns, retention, etc.) and use those when making decisions. Unfortunately, many companies still base their decisions very simplistic metrics and/or on "instinct".

Sorry, it was late and I didn't want to write a more substantive response.

The issue here is that politics are unavoidable. Being more data-driven is just another way of running your political process. And yes, it's a better way as long as you know its limitations. Collecting data and sifting through it to extract useful information takes time, creative thinking, and even "instinct" to figure out the right questions and hypotheses. Furthermore if you're going to collect data on dev workflow you better not have incentives there for employees or they will be gamed.

One of my pet peeves is technical people who worship so strongly at the altar of rationality that they are blind to their own biases. Even the most guileless and logical engineer still has an emotional life and worldview that forms the building blocks of what turns into "politics" when you get a large group of people together.

Re: Software Engineering at Google

#110
post #107
post #96

Not one word about what I consider the most toxic aspect of working at Google: blind allocation. Unless one absolutely does not care what one wishes to work on, joining Google is throwing your future into the Hogwarts hat of an ill-defined cabal of billionaires to pick your job at Google. Sometimes that works out, but most of the time you are allocated to whatever mission-critical project is currently leaking buttche…

Blind allocation? When I joined Google I was given 4 teams in 2 different locations to choose. I had lunch with each of the teams, and chose the one that better suited my style. To be fair, your experience at Google will depend a lot on what team you land in. Not everyone's given a choice, but now I know better: Talk to your recruiter and make them clear what you want.

Sure, that would be great, but as a result of leaving, I have been blacklisted by HR from ever returning. And the one guy that tried to bring me back in a year ago met in the lobby of his building and then walked me out to the marsh behind the GooglePlex before saying a word to me.

I then asked if I could use said technology for which I am a recognized expert, he said no, and that was the end of that.

I have been tempted to print myself a "Bad Cultural Fit - Google HR" T-shirt in response, but I have better things to do.

Post reply on HN