Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Is 2B Lines of Code, All in One Place

wired.com

Post reply on HN
Print view View original

141–150 of 347 posts

Re: Google Is 2B Lines of Code, All in One Place

#141

Post by kyrra »

Touche wrote: post #95
nevir wrote: post #90

Earlier quoted context omitted.

Being able to make sweeping changes to a shared piece of code, and ensure that everyone's up to date (Hi, Rosie!) and not broken by your change (yay TAP train!) is phenomenal as well.

How is this a side effect of it being in the same repository/

All dependencies between directories are specified as part of the bazel[0] BUILD files. Since dependencies between files/packages are all laid out in BUILD files, you can have a graph of all dependencies between all packages. So if you change something in 1 package it can figure out all packages that should be effected.

Bazel test[1] doesn't provide dependency testing as far as I know, but it creates the framework to support doing it.

[0] http://bazel.io/docs/build-encyclopedia.html

[1] http://bazel.io/docs/test-encyclopedia.html

Re: Google Is 2B Lines of Code, All in One Place

#143

Post by h1fra »

The comparison with windows really is just here to provide a something to compare for casual reader, it's not really that good. An OS is a huge project. But google has hundred of different project, apis, library, framework... Even unix with an "unlimited" source of developpers does not reach that point.

Re: Google Is 2B Lines of Code, All in One Place

#144

Post by ksk »

trhway wrote: post #137
ksk wrote: post #122

Earlier quoted context omitted.

I know that people have had experiences similar to yours. It's fine to disagree, but AFAIK pretty much all benchmarks show that there is no noticeable difference in performance from 7 to 8 to 10 and this matches with my own experience. I refuse to upgrade unless I get similar or better performance. But then again, I'm not really interested in researching every single benchmark. Windows is fast, stays fast, and that's…

>Windows is fast, stays fast, and that's pretty much all I care about. are you in CO?

what?

Re: Google Is 2B Lines of Code, All in One Place

#145

Post by packetslave »

Touche wrote: post #129
tuckerman wrote: post #105

Earlier quoted context omitted.

I think sometimes it's actually the opposite. I was able to make a breaking change to an API and update all of the callers to use the new one in a single commit. Tests would run on all the clients and, since in my workspace the server was updated simultaneously, I could be more sure it would work.

That's fine for small breaking changes like an API being renamed, but some times breaking changes require actually refactoring code which is hard/impossible to do without intimate knowledge of a codebase. Think python 2->3 or Angular 1->2. These types of changes do happen, and I bet they happen at Google. I don't think anyone is rewriting a downstream app when they make these changes. Most likely they are doing somet…

A talk and a paper on how we do large-scale refactoring in the C++ parts of the codebase

https://isocpp.org/blog/2015/05/cppcon-2014-large-scale-refa...

http://research.google.com/pubs/pub41342.html

Re: Google Is 2B Lines of Code, All in One Place

#146

Post by antics »

4ad wrote: post #132
antics wrote: post #119

Earlier quoted context omitted.

I can't say. I work here, but I don't speak for the company.

Well, you just said what it doesn't have, so I guess you speak for the company after all. Anyway, let me guess. Judging by how the size of all binaries shipped with Windows varied between releases, I'd be inclined to think Windows 10 does not have significantly more lines of code than Windows Vista. So I'd guess at most 100 million lines of code?

You are of course free to consult your employer and draw your line however you like.

For me, I'm comfortable saying that I don't speak for the company and leaving the numbers within an order of magnitude. When it becomes my job to decide which numbers are and aren't fit to talk about publicly, I'm happy to update you.

Re: Google Is 2B Lines of Code, All in One Place

#147

Post by hebecb »

durin42 wrote: post #89
sytse wrote: post #39

So a monolithic codebase makes it easier to make an organization wide change. Microservices make it easier to have people work and ship in independent teams. The interesting thing is that your can have have microservices with a monolithic codebase (as Google and Facebook are comprised of many services). But you can also have a monolithic service with many codebases (like our GitLab that uses 800+ gems that live in se…

Google has tons of services internally that talk via RPC. The monolithic repo means that it's much easier to hunt down and find people that are (say) using an outdated RPC method and help them fix their code. (Just one example of how it's useful even when things are mostly services.)

To add to your point: let's differentiate between a monolithic REPOSITORY vs a monolithic CODE BASE. Having all the code in a monolithic REPOSITORY allows developers to more efficiently explore and transform the code with a better understanding of the breadth of their impact.

Whether the CODE BASE is monolithic or not is orthogonal to the repository's nature. I was G for a couple years and I'd say they've done an ok job of breaking things up into libraries and services. Certainly there are interfaces that have done a better or worse job of setting the code up for open sourcing but because of the nature of the repository, large scale refactoring is more efficiently accomplished.

Re: Google Is 2B Lines of Code, All in One Place

#148

Post by nandhp »

yongjik wrote: post #70

One humorous side-effect of having all that code viewable (and searchable!) by everyone was that the codebase will contain whatever typo, error, or mistake you can think of (and convert into a regular expression). I remember seeing an internal page with dozens of links for humorous searches like "interger", "funciton", or "([A-Z][a-z]+){7,} lang:java"...

And then you killed off the public version and you keep that fun (and useful) toy to yourself.

(But as great as Google Code Search was, my grudge is because of Reader.)

Re: Google Is 2B Lines of Code, All in One Place

#149

Post by cpeterso »

jpollock wrote: post #31
jfkw wrote: post #27

How do the monolithic repository companies handle dependencies on external source code? Are libraries and large projects e.g. RDBMS generally vendored/forked into the monolithic repositories, regardless of whether the initial intent is to make significant changes?

There's typically a subdirectory called third_party, with subdirectories for each vendor, product and version. If the team is smart, they will also enact a rule saying "only one version". If you're really, really smart, local changes are kept as a set of patches, keeping them separate from the imported tar file. So, for source deliveries: third_party/apache/httpd/2.4/release.tgz /patch.tgz /Makefile (or other config)…

For example, here is Chromium's third_party directory:

https://chromium.googlesource.com/chromium/src.git/+/master/...

Re: Google Is 2B Lines of Code, All in One Place

#150

Post by jakub_g »

Some questions that immediately come to my mind:

- What is the disk size of a shallow clone of a repo (without history)?

- Can each developer actually clone the whole thing, or you do partial checkout?

- Does the VCS support a checkout of a subfolder (AFAIK mercurial, same as git, does not support it)?

- How long does it take to clone the repo / update the repo in the morning?

Since people are talking about huge across-repo refactorings, I guess it must be possible to clone the whole thing.

Facebook faces similar issues as Google with scaling so they wrote some mercurial extensions, e.g. for cloning only metadata instead of whole contents of each commit [1]. Would be interesting to know what Google exactly modified in hg.

[1] https://code.facebook.com/posts/218678814984400/scaling-merc...

Post reply on HN