Live data from Hacker News

Google Is 2B Lines of Code, All in One Place

wired.com

21–30 of 347 posts

Re: Google Is 2B Lines of Code, All in One Place

#22

I will say that I saw and experienced many things that changed my definition of 'large' at Google, but the most amazing was the source code control / code review / build system that kept it all together. The bad news was that it allowed people to say "I've just changed the API to to support the initiative, code released after this commit will need to be updated." and have that effect hundreds of projects, but at the…

The solution to the excessive API change problem is to force whoever changes the API to fix all the consumers himself before the change is accepted.

The Linux kernel generally uses this policy for internal APIs for example.

Re: Google Is 2B Lines of Code, All in One Place

#23
post #13

"The two internet giants (Google and Facebook) are working on an open source version control system that anyone can use to juggle code on a massive scale. It’s based on an existing system called Mercurial. “We’re attempting to see if we can scale Mercurial to the size of the Google repository,” Potvin says, indicating that Google is working hand-in-hand with programming guru Bryan O’Sullivan and others who help overs…

Because Google and Facebook are using Mercurial over Git internally. Edit: And for those that are just shocked that git isn't the answer. Facebook: https://code.facebook.com/posts/218678814984400/scaling-merc... Google: http://www.primordia.com/blog/2010/01/23/why-google-uses-mer...

Well, Piper conforms to the Perforce API-ish, and Android and Chrome are both on Git.

Mercurial was pushed internally as being the "better" (for some dimension of better) between it and Git back in 2010, but I think even the most hardline Mercurial fans have realized that in order to meet developers in the middle in 2015, we need to use Git for our open-source releases. We have a large investment in Gerrit [1] and Github [2] now.

So the Mercurial comment is probably entirely based on scaling and replacement for the Piper Perforce API, rather than anything externally facing.

[1] https://www.gerritcodereview.com/ [2] https://github.com/google

Re: Google Is 2B Lines of Code, All in One Place

#24
post #22

I will say that I saw and experienced many things that changed my definition of 'large' at Google, but the most amazing was the source code control / code review / build system that kept it all together. The bad news was that it allowed people to say "I've just changed the API to to support the initiative, code released after this commit will need to be updated." and have that effect hundreds of projects, but at the…

The solution to the excessive API change problem is to force whoever changes the API to fix all the consumers himself before the change is accepted. The Linux kernel generally uses this policy for internal APIs for example.

We do, mostly. Because Piper is a global repository, we have systems to do global safe refactors, and do so often. If the API changes drastically, there's usually a lengthy deprecation period before the API is switched over.

Re: Google Is 2B Lines of Code, All in One Place

#25

"The two internet giants (Google and Facebook) are working on an open source version control system that anyone can use to juggle code on a massive scale. It’s based on an existing system called Mercurial. “We’re attempting to see if we can scale Mercurial to the size of the Google repository,” Potvin says, indicating that Google is working hand-in-hand with programming guru Bryan O’Sullivan and others who help overs…

Why not IBM? Why not Microsoft? Is that what the HN crowd has turned into? You now have to justify the non mainstream solution like a scared mega-corp career drone who will only go with the safest bet?

Git isn't so awesome that it's inconceivable that people would be willing to use something else.

Re: Google Is 2B Lines of Code, All in One Place

#26
post #22

I will say that I saw and experienced many things that changed my definition of 'large' at Google, but the most amazing was the source code control / code review / build system that kept it all together. The bad news was that it allowed people to say "I've just changed the API to to support the initiative, code released after this commit will need to be updated." and have that effect hundreds of projects, but at the…

The solution to the excessive API change problem is to force whoever changes the API to fix all the consumers himself before the change is accepted. The Linux kernel generally uses this policy for internal APIs for example.

I guess that works for the Linux kernel, but I would presume that for a large distributed operation like Google it would be much better to simply deprecate/version APIs and have the project teams update to a deadline.

I mean, it's presumably impossible to have a single computer running a single OS build all of the Google software and run the testing.

Re: Google Is 2B Lines of Code, All in One Place

#27
How do the monolithic repository companies handle dependencies on external source code?

Are libraries and large projects e.g. RDBMS generally vendored/forked into the monolithic repositories, regardless of whether the initial intent is to make significant changes?

Re: Google Is 2B Lines of Code, All in One Place

#28
post #2

Those are mind-boggling numbers. Although I kind of doubt that "almost every" engineer has access to the entire repo, especially when it comes to the search ranking stuff.

FWIW, apart from the previously mentioned sensitive stuff, we give engineering interns the same level of access we give full-time engineers. We keep things open because it makes things faster; we have an excellent code search tool that's great for navigating through the Piper repo (e.g. finding subclasses, finding uses of an API) which really speeds up dev time.

When we're not talking about the sensitive stuff, there's not much magic to what many engineers write every day, it's the same "glue technology X to technology Y" stuff you see everywhere, so I don't think there's any value to hiding that in the name of secrecy.

Re: Google Is 2B Lines of Code, All in One Place

#29
In the spirit of "You didn't build that," I wonder how many lines of code comprise the binaries that Google binaries run on? Windows, Linux, network stacks, Mercurial, etc, etc.

I also wonder if there's a circular relationship anywhere in there.

Post reply on HN