Looks like someone's going to have to update this: http://www.informationisbeautiful.net/visualizations/million...
Google Is 2B Lines of Code, All in One Place
71–80 of 347 posts
Re: Google Is 2B Lines of Code, All in One Place
#72I will say that I saw and experienced many things that changed my definition of 'large' at Google, but the most amazing was the source code control / code review / build system that kept it all together. The bad news was that it allowed people to say "I've just changed the API to to support the initiative, code released after this commit will need to be updated." and have that effect hundreds of projects, but at the…
> With the orb on their desk telling them at that their integration and unit tests were passing. Anymore info on this? Is it in house hardware?
Started with a project that used the Ambient Orb to show test status, they were expensive which led to a contest to make the cheapest one, basically an ATTiny with a software USB hid stack tied to an RGB LED. Henner Zeller won that contest and I think maybe 5000 gOrbs were built.
Re: Google Is 2B Lines of Code, All in One Place
#73Earlier quoted context omitted.
I guess that works for the Linux kernel, but I would presume that for a large distributed operation like Google it would be much better to simply deprecate/version APIs and have the project teams update to a deadline. I mean, it's presumably impossible to have a single computer running a single OS build all of the Google software and run the testing.
Google is rumored to have an extremely powerful distributed compilation farm. It wouldn't surprise me if a single developer could make a change that affects the entire codebase and test it themselves.
The key to this in her talk was TAP
Re: Google Is 2B Lines of Code, All in One Place
#74"The two internet giants (Google and Facebook) are working on an open source version control system that anyone can use to juggle code on a massive scale. It’s based on an existing system called Mercurial. “We’re attempting to see if we can scale Mercurial to the size of the Google repository,” Potvin says, indicating that Google is working hand-in-hand with programming guru Bryan O’Sullivan and others who help overs…
Because Google and Facebook are using Mercurial over Git internally. Edit: And for those that are just shocked that git isn't the answer. Facebook: https://code.facebook.com/posts/218678814984400/scaling-merc... Google: http://www.primordia.com/blog/2010/01/23/why-google-uses-mer...
There are also a massive number of git repos internal to googl e as well.
One thing mentioned in the paper, but not mentioned here, is that there are teams that live in GIT, such as android and chrome, but they are not monolithic.
Re: Google Is 2B Lines of Code, All in One Place
#75I wonder how close the "piper" system is to the code.google.com project.
I worked on code.google.com, i can tell you the are 100% unrelated.
piper grew out of a need to scale the source control system the initial internal repositories were using
code.google.com was a completely separate thing supporting completely different version control models, and a very different scale (very large number of small repositories, vs very small number of very large repositories)
Re: Google Is 2B Lines of Code, All in One Place
#76This hurts just thinking about what the build, test and deploy systems must look like.
Re: Google Is 2B Lines of Code, All in One Place
#77Earlier quoted context omitted.
Because Google and Facebook are using Mercurial over Git internally. Edit: And for those that are just shocked that git isn't the answer. Facebook: https://code.facebook.com/posts/218678814984400/scaling-merc... Google: http://www.primordia.com/blog/2010/01/23/why-google-uses-mer...
I was at Google for a long time and it was almost entirely Perforce (some Googlers used git) until it was Piper (which is very similar to Perforce). L&S used CVS at Stanford for the original Backrub/Google code and for some time after the establishment of Google, Inc.
Is Piper a wrapper around Perforce or built from scratch? What is it exactly, if you don't mind me asking?
Re: Google Is 2B Lines of Code, All in One Place
#78One humorous side-effect of having all that code viewable (and searchable!) by everyone was that the codebase will contain whatever typo, error, or mistake you can think of (and convert into a regular expression). I remember seeing an internal page with dozens of links for humorous searches like "interger", "funciton", or "([A-Z][a-z]+){7,} lang:java"...
> "([A-Z][a-z]+){7,} lang:java
Yeah this one was my favorite of the code search examples, there are some really good ones in there.
Re: Google Is 2B Lines of Code, All in One Place
#79Earlier quoted context omitted.
> With the orb on their desk telling them at that their integration and unit tests were passing. Anymore info on this? Is it in house hardware?
IIRC, it was built on a hacked up Ambient Orb http://ambientdevices.myshopify.com/products/stock-orb
Wow, $150? Shocked there isn't a cheaper USB version that is more easily hackable. I mean, is it really more than color-changing LEDs in a plastic sphere?
Re: Google Is 2B Lines of Code, All in One Place
#80Comparing "Google" to Windows isn't really a fair comparison. I'm sure all of the code that represents products that Microsoft has in the wild far exceeds 2B lines.
Note that this is just the monolithic repository.
Google also has other non-piper repositories containing hundreds of millions of lines too :P
For example, android and chrome are git based.
Note also that when codesearch used to crawl and index the world's code, it was not actually that large. It used to download and index tarballs, svn and cvs repositories, etc.
All told, the amount of code in the world that it could find on the internet a few years ago was So while you may be right or wrong, i don't think it's as obvious you are right as you do.