The worrying point here is the checkout of 8GB as opposed to the history size itself (46GB). If git is fast enough with SSD, this is hardly anything to worry about. I actually prefer monolithic repos (I realize that the slide posted might be in jest). I have seen projects struggle with submodules and splitting up modules into separate repos. People change something in their module. They don't test any upstream module…
How does monolithic repos solve that. Surely people who fix bugs in a library aren't testing the entirety of Facebook every time (how long would that even take? Assuming they've even set such a thing up.)
In terms of engineering tradeoffs, this reminds me of a recent talk by Alan Kay where he says that to build the software of the future, you have to pay extra to get the hardware of the future today. [1] Joel Spolsky called it "throwing money at the problem" when, five years ago he got SSD's for everybody at Fog Creek just to deal with a slow build. [2] I don't use Facebook, and I'm not suggesting that they're buildin…
Facebook tends to throw engineer time at the problem, though. I know one Facebook DevCon I went to they presented how they completely wrote their own build system because Ant was too slow for them.
Having used ant as a build system for Android projects, I don't blame them.
In my admittedly limited experience (Windows 7 x64, ant, Android SDK) ant is terribly slow to build projects with multiple source library dependencies and throwing hardware at the problem doesn't speed it up that much.
Is there a reason why they keep everything in the same repo? Can’t you just split the code across multiple smaller repos?
When you split code across smaller repos, you gain a different problem: version management and potentially dependencies using different versions of the same project.
I bet most of that size is made up from the various dependencies Facebook probably has, though I'm still surprised it's that large. I expected the background worker things, like the facial recognition system for tagging people, and the video re-encoding libs, to be housed on separate repositories. I also wonder if that size includes a snapshot of a subset of Facebook's Graph, so that each developer has a "mini-facebo…
Having all code in a single repository increases developer productivity by lowering the barrier to change. You can make a single atomic commit in one repository as opposed to N commits in M repositories. This is much, much easier than dealing with subrepos, repo sync, etc. Unified repos scales well up to a certain point before troubles arise. e.g. fully distributed VCS starts to break down when you have hundreds of M…
Hello there, have you heard of service oriented architecture? You must be joking to justify a single repository with "easier to change". Your problem is that the code base must be tightly coupled if splitting the services out to different repos is not possible and you need to contribute to multiple repositories to get something done. I would say, the biggest change in Amazon's architecture was moving over to the service oriented way and it was worth the effort. Developers are forced to separate different functions to separate services and they are in charge of that service. If it goes down their are getting the alerts. All of the services are using 250ms timeouts so there is no cascading effect when a services goes down. The web page consists of few thousand service calls and it degrades gracefully. Facebook obviously have some tech depth that they need to fix. Using stupid design justified with some random crap that does not even make sense is not really acceptable (at least for me).
I think they did, have heard similar figures from them before, all deployed as one file. Macroservices...
unlikely. they probably have a very dirty repo with tons of binaries, images, blah blah blah. It's highly unlikely they actually wrote 8GB of code, and the 46GB .git directory will be littered with binary blob changes, etc. This is really just to "impress" two people: 1) People who love Facebook 2) People who don't know anything about version control and/or how to do proper version control (no binaries in the scm).
The worrying point here is the checkout of 8GB as opposed to the history size itself (46GB). If git is fast enough with SSD, this is hardly anything to worry about. I actually prefer monolithic repos (I realize that the slide posted might be in jest). I have seen projects struggle with submodules and splitting up modules into separate repos. People change something in their module. They don't test any upstream module…
How does monolithic repos solve that. Surely people who fix bugs in a library aren't testing the entirety of Facebook every time (how long would that even take? Assuming they've even set such a thing up.)
It is at least easier to correlate the changes. When you have X+ modules, you have potentially X+ histories you have to look at to know when a change was seen with another change.
Yes, and an intern would be subject to a code review before pushing to master - no way would they have write access to the "master" repo.
Yes, but the new intern would be able to read all the source and "secret sauces". I doubt that an intern on Google would've access to the search codebase. I'd wager that only a handful of trusted employees have access to that codebase.
All Google code is stored in one master repository, for all products. An intern can look at the search code.
Yes, and an intern would be subject to a code review before pushing to master - no way would they have write access to the "master" repo.
Yes, but the new intern would be able to read all the source and "secret sauces". I doubt that an intern on Google would've access to the search codebase. I'd wager that only a handful of trusted employees have access to that codebase.
Pay attention to the footnote: *8 GB plus 46 GB .git directory
8 GB is still a lot. Would be interesting to know how much of it is actual code and how much is just images and so on.
8GB would be at least 100 million lines of code (upper bound with 80 characters per each line). For comparison Linux has 15+ million lines of code, PHP 2+M.