Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software Engineering at Google

arxiv.org

Post reply on HN
Print view View original

41–50 of 161 posts

Re: Software Engineering at Google

#41

Post by mikekchar »

throwawayGogEng wrote: post #35
__strisk wrote: post #29

Is the practice of shoving all disparate pieces of proprietary software (or individual projects) in the same repo a common occurrence? I have found that pulling unrelated changes just so that I can push my changes is an inconvenience. Furthermore, tracking the history of a particular project is confounded due to unrelated commits. I am sure that their vcs (piper?) makes this a feasible task, but for git, it seems lik…

Coming from companies that use reasonably-sized git repos, I absolutely hated Google's VCS. Here's some of my painpoints with it: * No branches. If you want to make a temporary code branch, you create a CL (Google's version of a pull request), but never submit it. This means nobody else can collaborate on it with you, and it must be manually updated to HEAD. * No CL collaboration. Unlike Git branches, CLs can only co…

It's interesting. This is exactly how it's worked at every large company I've been at. You have these enormous source trees that are essentially unbranchable. Then you have load build teams that build the latest checked in version. When you want to do anything that uses a component you aren't working on, you grab the built version and use it locally. They you hope like crazy that any changes you make don't break any downstream users (who are integrating with yesterday's build). If the build breaks (which happens frequently), then people use builds from 1, 2, or 3 days ago, leading to inevitable "integration hell".

Although I seriously doubt that Google has this problem, one of the biggest drawbacks of the scheme is that nobody knows how to build anything -- not even their own project. If you are coordinating with another project, then you're always having to wait days for the load build to finish so you can get their changes. If the build breaks for an unrelated reason, you can lose a whole week of development to screwing around.

When working in that kind of environment, I tend to maintain my own VCS, then squash my commits when integrating with the main VCS. I also do all my own load build. Everywhere I've worked, I've been heavily criticised (usually by management) for doing this, but productivity has been on my side, so people reluctantly accept it. I often find it strange how so many people prefer doing it the other way...

Re: Software Engineering at Google

#43

Post by bskap »

pzh wrote: post #8
gumby wrote: post #3

Earlier quoted context omitted.

> What the heck is the other 991000000? Says right in the article: various config and dependency files, presumably both as caches (where everyone would generate the same product) or as a record of where things stood on at time t. For example: > In some cases, notably Go programs, build files can be generated (and updated) automatically, since the dependency information in the BUILD files is (often) an abstraction of…

So basically somebody can write a script to put this Build file in gitignore, save the company millions of dollars, and get promoted for it?

Nah, because it would cost the company billions of dollars in lost productivity waiting for these files to get re-processed every time someone built the thing. Google's general philosophy is that humans are expensive and computers are cheap, so pretty much anything that helps the humans go faster is a going to be a net benefit in the long run.

Re: Software Engineering at Google

#44

Post by theparanoid »

richsinn wrote: post #23
aanm1988 wrote: post #2

They have a billion files in their repo, 9 million are source files. What the heck is the other 991000000? I skimmed this. Mostly just stuff any competent company would/should be doing. it's google though, so they act like it's super awesome.

> Mostly just stuff any competent company would/should be doing. it's google though, so they act like it's super awesome. Yes, you're absolutely correct. But here's the thing - it was actually Google that pioneered many of this. Many of the big/competent companies that are following these practices are because of Google's "DNA" leaking into those companies (via former employees bringing along the best practices learn…

Microsoft used to have the best practices and...they were mostly as good as Google. Everything old is new again.

Re: Software Engineering at Google

#45

Post by bskap »

sjakobi wrote: post #26

Why aren't OKR scores used as an input to performance appraisals?

Because they aren't measures of how good you are at developing, testing, and releasing software, they're just measures of how good you are at estimating the amount of work it'll take to complete a project 3 months in advance.

Re: Software Engineering at Google

#46

Post by mck- »

A lot of the things they had to build in-house for repo/build/test/deploy as described in chapter two, all of us are fortunate enough to get for (almost) free with all the tools these days.

It's a good time to be a founder :)

Re: Software Engineering at Google

#48

Post by throwawayGogEng »

jmillikin wrote: post #37
throwawayGogEng wrote: post #35

Earlier quoted context omitted.

Coming from companies that use reasonably-sized git repos, I absolutely hated Google's VCS. Here's some of my painpoints with it: * No branches. If you want to make a temporary code branch, you create a CL (Google's version of a pull request), but never submit it. This means nobody else can collaborate on it with you, and it must be manually updated to HEAD. * No CL collaboration. Unlike Git branches, CLs can only co…

Everything you posted is wrong, which makes me believe you know everything you posted is wrong. Nobody writes things that inaccurate by accident. For the benefit of YC, here are corrections: > No branches. If you want to make a temporary code branch, > you create a CL (Google's version of a pull request), but > never submit it. This means nobody else can collaborate on > it with you, and it must be manually updated t…

Thanks for the info!

> Piper has branches

I've worked on two teams so far, neither of which used the branches you described. They opted to either flag off features or keep long-running CLs.

However, I'll try to learn more about Piper branches. I doubt my team will make large workflow shifts, but it would still be good to understand!

> CLs are equivalent to Git commits

I'd argue that CLs are not equivalent to Git commits, the equivalent would be a CL snapshot. Best practices in git are to have frequent, small commits. CLs tend to be much larger, and the review process means that having small CLs would greatly slow workflow.

> There is a global testing system for the entire repository, which is used to decide whether a particular project's tests pass. Commits on which all relevant tests pass are the branch point for releases.

Yes, but this doesn't help with development. When HEAD is broken, a developer has to chose between developing on an outdated codebase, or developing on a broken codebase.

> Many third-party open-source libraries have multiple versions, and new upstream releases are added when there's either a security/bug fix, or someone wants a new feature.

Popular libraries may be updated more often, however other libraries don't have many resources dedicated to them. After a brief correspondence with the team that manages third-party libraries, I decided it would be easier to implement the feature myself instead of following whatever process was required to update the library. And no, I wasn't trying to use 2 versions of the same library.

Despite your assertion, I'm not trying to write anything incorrect, and I appreciate your response.

Re: Software Engineering at Google

#49

Post by milesrout »

__strisk wrote: post #29

Is the practice of shoving all disparate pieces of proprietary software (or individual projects) in the same repo a common occurrence? I have found that pulling unrelated changes just so that I can push my changes is an inconvenience. Furthermore, tracking the history of a particular project is confounded due to unrelated commits. I am sure that their vcs (piper?) makes this a feasible task, but for git, it seems lik…

No, it's not common. It's something that a couple of big companies have done, and while it might work for them it does NOT work for most people. It's a really, really bad way of doing development.

Re: Software Engineering at Google

#50

Post by throwawayGogEng »

mikekchar wrote: post #41
throwawayGogEng wrote: post #35

Earlier quoted context omitted.

Coming from companies that use reasonably-sized git repos, I absolutely hated Google's VCS. Here's some of my painpoints with it: * No branches. If you want to make a temporary code branch, you create a CL (Google's version of a pull request), but never submit it. This means nobody else can collaborate on it with you, and it must be manually updated to HEAD. * No CL collaboration. Unlike Git branches, CLs can only co…

It's interesting. This is exactly how it's worked at every large company I've been at. You have these enormous source trees that are essentially unbranchable. Then you have load build teams that build the latest checked in version. When you want to do anything that uses a component you aren't working on, you grab the built version and use it locally. They you hope like crazy that any changes you make don't break any…

> If the build breaks for an unrelated reason, you can lose a whole week of development to screwing around.

This happens often on some projects at Google (although 2 days is the longest I've seen it broken). Others have told me they use this time for documenting code and writing design-documents.

> When working in that kind of environment, I tend to maintain my own VCS, then squash my commits when integrating with the main VCS.

Google actually has a tool that allows developers to use Git on their local machine, which is squashed into a CL when pushed. However, some projects are too reliant on the old system for this to work.

Post reply on HN