Although this is large for a company that deals mostly in web-based projects, it's nothing compared to repository sizes in game development. Usually game assets are in one repository (including compiled binaries) and code in another. The repository containing the game itself can grow to hundreds of gigabytes in size due to tracking revision history on art assets (models, movies, textures, animation data, etc). I woul…
But they surely don't use git for that, right? In scenarios like this a versioning system that does not track all history locally would be a better fit.
Facebook's git repo is 54GB
101–110 of 245 posts
Re: Facebook's git repo is 54GB
#102The worrying point here is the checkout of 8GB as opposed to the history size itself (46GB). If git is fast enough with SSD, this is hardly anything to worry about. I actually prefer monolithic repos (I realize that the slide posted might be in jest). I have seen projects struggle with submodules and splitting up modules into separate repos. People change something in their module. They don't test any upstream module…
>People change something in their module. They don't test any upstream modules because it's not their problem anymore. If you use any sort of versioning this shouldn't ever cause a problem.
Re: Facebook's git repo is 54GB
#103Earlier quoted context omitted.
Yes, and an intern would be subject to a code review before pushing to master - no way would they have write access to the "master" repo.
Yes, but the new intern would be able to read all the source and "secret sauces". I doubt that an intern on Google would've access to the search codebase. I'd wager that only a handful of trusted employees have access to that codebase.
Re: Facebook's git repo is 54GB
#104Earlier quoted context omitted.
Honestly, I prefer check-in everything shops. It's way too often otherwise some different Java version, or IDE version, or Maven central being down screws something up, or you have to wait a long time for a Chef recipe or disk image to give you the reference version. Half my day today was dealing with someone updating Java on half our continuous build system slave computers and breaking everything because it didn't h…
That sounds like either poor Sys Admin'ing and/or poor documentation... SCM should not have "clutter" in it, otherwise you wind up with an all-day download of 54GB of dead or useless garbage. The Kernel's repo is only a few GB's and it has MANY more changes and much more history than FB does...
Re: Facebook's git repo is 54GB
#105The worrying point here is the checkout of 8GB as opposed to the history size itself (46GB). If git is fast enough with SSD, this is hardly anything to worry about. I actually prefer monolithic repos (I realize that the slide posted might be in jest). I have seen projects struggle with submodules and splitting up modules into separate repos. People change something in their module. They don't test any upstream module…
>People change something in their module. They don't test any upstream modules because it's not their problem anymore. If you use any sort of versioning this shouldn't ever cause a problem.
Re: Facebook's git repo is 54GB
#106It appears they are now using Mercurial and working on scaling that (also noted by several others in this discussion): https://code.facebook.com/posts/218678814984400/scaling-merc...
Re: Facebook's git repo is 54GB
#107Earlier quoted context omitted.
It should be possible to restrict each employee's access to specific parts of the repository. However, I can't really see Facebook doing that. Everyone having access to everything must be worth the security trade-off. On the other hand, I suppose it's debatable whether it would be a trade-off at all.
>Everyone having access to everything must be worth the security trade-off. I would find this extremely hard to believe, especially at Facebook. At any software company, your code base is what defines you as a company; there is no way they'd let the good stuff sneak out like that.
Re: Facebook's git repo is 54GB
#108In terms of engineering tradeoffs, this reminds me of a recent talk by Alan Kay where he says that to build the software of the future, you have to pay extra to get the hardware of the future today. [1] Joel Spolsky called it "throwing money at the problem" when, five years ago he got SSD's for everybody at Fog Creek just to deal with a slow build. [2] I don't use Facebook, and I'm not suggesting that they're buildin…
Facebook tends to throw engineer time at the problem, though. I know one Facebook DevCon I went to they presented how they completely wrote their own build system because Ant was too slow for them.
Re: Facebook's git repo is 54GB
#109Earlier quoted context omitted.
It should be possible to restrict each employee's access to specific parts of the repository. However, I can't really see Facebook doing that. Everyone having access to everything must be worth the security trade-off. On the other hand, I suppose it's debatable whether it would be a trade-off at all.
>Everyone having access to everything must be worth the security trade-off. I would find this extremely hard to believe, especially at Facebook. At any software company, your code base is what defines you as a company; there is no way they'd let the good stuff sneak out like that.
Now what? How do you get users? "We are just like facebook - only your friends aren't here" probably wouldn't get users excited.
And if you somehow DID manage to get users, don't you think there are "watermarks" in the code, that they could detect and sue you to death with?
Re: Facebook's git repo is 54GB
#110Earlier quoted context omitted.
Indeed the process of facebook code deployment is pretty complex: https://www.facebook.com/publications/514128035341603/ And I assume they have extra requirements for interns to push code.
that is a fascinating article, thank you. i like the idea of 'push karma'.
I liked this idea too : "all engineers who contributed code must be available online during the push. The release system verifies this by contacting them automatically using a system of IRC bots; if an engineer is unavailable (at least for daily pushes), his or her commit will be reverted."
That way, they are be able to react very quickly in case of a problem.