Live data from Hacker News

Software Engineering at Google

arxiv.org

31–40 of 161 posts

Re: Software Engineering at Google

#31
post #2

They have a billion files in their repo, 9 million are source files. What the heck is the other 991000000? I skimmed this. Mostly just stuff any competent company would/should be doing. it's google though, so they act like it's super awesome.

Translation files, xml files, some data files, images, and lots of other things.

Ps: goog employee

Re: Software Engineering at Google

#32
post #2

They have a billion files in their repo, 9 million are source files. What the heck is the other 991000000? I skimmed this. Mostly just stuff any competent company would/should be doing. it's google though, so they act like it's super awesome.

[deleted]

Re: Software Engineering at Google

#35

Is the practice of shoving all disparate pieces of proprietary software (or individual projects) in the same repo a common occurrence? I have found that pulling unrelated changes just so that I can push my changes is an inconvenience. Furthermore, tracking the history of a particular project is confounded due to unrelated commits. I am sure that their vcs (piper?) makes this a feasible task, but for git, it seems lik…

Coming from companies that use reasonably-sized git repos, I absolutely hated Google's VCS.

Here's some of my painpoints with it:

* No branches. If you want to make a temporary code branch, you create a CL (Google's version of a pull request), but never submit it. This means nobody else can collaborate on it with you, and it must be manually updated to HEAD.

* No CL collaboration. Unlike Git branches, CLs can only contain changes from one user.

* No stable branch. Since everything is essentially on one long branch, it's a real hassle when a project is broken at HEAD. Sure, integration tests should ideally prevent this. In practice, HEAD is often broken. Teams have created bash scripts and mailing lists to determine 'stable' old versions that can be checked out for development.

* Single versions of libraries. Any library that is used is also checked into the VCS. However, only one version of the library can exist in the codebase, which is rarely updated. However, there are exceptions to this.

At one point, Sergey mentioned bringing Google "up to industry standards" regarding VCS's. However, that would be a monumental task and I doubt it will happen.

Re: Software Engineering at Google

#37

Is the practice of shoving all disparate pieces of proprietary software (or individual projects) in the same repo a common occurrence? I have found that pulling unrelated changes just so that I can push my changes is an inconvenience. Furthermore, tracking the history of a particular project is confounded due to unrelated commits. I am sure that their vcs (piper?) makes this a feasible task, but for git, it seems lik…

Coming from companies that use reasonably-sized git repos, I absolutely hated Google's VCS. Here's some of my painpoints with it: * No branches. If you want to make a temporary code branch, you create a CL (Google's version of a pull request), but never submit it. This means nobody else can collaborate on it with you, and it must be manually updated to HEAD. * No CL collaboration. Unlike Git branches, CLs can only co…

Everything you posted is wrong, which makes me believe you know everything you posted is wrong. Nobody writes things that inaccurate by accident.

For the benefit of YC, here are corrections:

  > No branches. If you want to make a temporary code branch,
  > you create a CL (Google's version of a pull request), but
  > never submit it. This means nobody else can collaborate on
  > it with you, and it must be manually updated to HEAD.
Piper has branches, which can be committed to as normal by any number of engineers. It's common to have both branches for developing certain features ("dev branches"), and branches pinned to a stable base version with cherry-picked bug fixes ("release branches").

  > No CL collaboration. Unlike Git branches, CLs can only
  > contain changes from one user.
CLs are equivalent to Git commits. Collaboration is expected to occur via a series of CLs, just like Git-based projects have large changes made via a series of smaller commits.

  > No stable branch. Since everything is essentially on one
  > long branch, it's a real hassle when a project is broken
  > at HEAD. Sure, integration tests should ideally prevent
  > this. In practice, HEAD is often broken. Teams have
  > created bash scripts and mailing lists to determine
  > 'stable' old versions that can be checked out for
  > development.
There is a global testing system for the entire repository, which is used to decide whether a particular project's tests pass. Commits on which all relevant tests pass are the branch point for releases. This is similar to the Linux kernel's dev model, where stable releases are cut at known-healthy points in an evolving codebase.

Important libraries define more rigorous releases, similar to Git labels, which are updated automatically every day or two. These both reduce the amount of tests that need to run, and reduce chances of errors in low-level code affecting many teams.

  > Single versions of libraries. Any library that is used is
  > also checked into the VCS. However, only one version of
  > the library can exist in the codebase, which is rarely
  > updated. However, there are exceptions to this.
Many third-party open-source libraries have multiple versions, and new upstream releases are added when there's either a security/bug fix, or someone wants a new feature.

Three of the four languages most often used at Google (Python, C++, Go) do not allow multiple versions of a library to be linked into a single process due to symbol conflicts. This is a limitation of those languages, not of the Google repository, and they affect any company that allows use of third-party code. The standard recommendation at Google is to avoid dependency hell by sharding large binaries and using RPCs to communicate. This development model has many advantages that have been documented elsewhere.

Re: Software Engineering at Google

#38

Is the practice of shoving all disparate pieces of proprietary software (or individual projects) in the same repo a common occurrence? I have found that pulling unrelated changes just so that I can push my changes is an inconvenience. Furthermore, tracking the history of a particular project is confounded due to unrelated commits. I am sure that their vcs (piper?) makes this a feasible task, but for git, it seems lik…

There are advantages and disadvantages. Here's an article about the good parts:

https://medium.freecodecamp.com/how-google-builds-a-web-fram...

Re: Software Engineering at Google

#39
>>Engineers are permitte d to spend up to 20% of their time working on any project of their choice, without needing approval from their manager or anyone else.

Can someone tell more about what they did and the things are that permitted (even though without needing approval.

Re: Software Engineering at Google

#40

>>Engineers are permitte d to spend up to 20% of their time working on any project of their choice, without needing approval from their manager or anyone else. Can someone tell more about what they did and the things are that permitted (even though without needing approval.

It's all kinda of things: I did mine on another team I was thinking of transferring to (as a sort of pilot program), a friend of mine in ads worked on a cloud robotics team (this was like 5 years ago), another friend spent some time on a research team for the stuff he had done his thesis on, etc
Post reply on HN