Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook's git repo is 54GB

twitter.com

Post reply on HN
Print view View original

61–70 of 245 posts

Re: Facebook's git repo is 54GB

#61

Post by rwmj »

infogulch wrote: post #22
VikingCoder wrote: post #2

Pay attention to the footnote: *8 GB plus 46 GB .git directory

The new depth option [1] on git clone as of 1.9.x should make the .git dir much more reasonable. > --depth : Create a shallow clone with a history truncated to the specified number of revisions. If the OS and applications that are creating and modifying files are all honest about the file dates, it should be possible to only scan dates instead of reading out every file. Or even use ionotify-like events to track what…

I've wanted this option for so long :-) Are there any downsides to it, apart from the obvious one of 'git log' not going all the way back in time? I mean, do any commands break unexpectedly?

Re: Facebook's git repo is 54GB

#62

Post by SnakeDoc »

lnanek2 wrote: post #58
SnakeDoc wrote: post #47

I hope everyone realizes this is not 54GB of code, but in fact, is more likely a very public showing of very poor SCM management. They likely have tons of binaries in there, many many full codebase changes (whitespace, tabs, line endings, etc). Also not to mention how much dead code lives in there?

Honestly, I prefer check-in everything shops. It's way too often otherwise some different Java version, or IDE version, or Maven central being down screws something up, or you have to wait a long time for a Chef recipe or disk image to give you the reference version. Half my day today was dealing with someone updating Java on half our continuous build system slave computers and breaking everything because it didn't h…

That sounds like either poor Sys Admin'ing and/or poor documentation... SCM should not have "clutter" in it, otherwise you wind up with an all-day download of 54GB of dead or useless garbage. The Kernel's repo is only a few GB's and it has MANY more changes and much more history than FB does...

Re: Facebook's git repo is 54GB

#63

Post by slig »

TD-Linux wrote: post #50
slig wrote: post #41

Am I missing something or this means a new intern working on a small feature, for instance, would have access to entire codebase?

Yes, and an intern would be subject to a code review before pushing to master - no way would they have write access to the "master" repo.

Yes, but the new intern would be able to read all the source and "secret sauces".

I doubt that an intern on Google would've access to the search codebase. I'd wager that only a handful of trusted employees have access to that codebase.

Re: Facebook's git repo is 54GB

#64

Post by natrius »

ianphughes wrote: post #28
natrius wrote: post #19

Earlier quoted context omitted.

Dear everyone: you should be using Phabricator. It is Facebook's collected wisdom about software development expressed in software. It has improved my life substantially. The code review is better than Github's, and their linear, squashed commit philosophy has worked out much better than the way I used to do things.

It looks pretty great. How does it compare to Atlassian products (if you have used any)?

I've used JIRA, and I slightly prefer Phabricator's tasks. They're mostly the same.

Re: Facebook's git repo is 54GB

#65

Post by SnakeDoc »

indygreg2 wrote: post #17
ianphughes wrote: post #12

I wonder what their branching strategy is like and how merges are gated with a single codebase of that size?

They aim for a completely linear history. They may even have a policy of not allowing merge commits. It is described in various places on the internet. I like https://secure.phabricator.com/book/phabflavor/article/recom... because it and its sister articles on code review and revision control are terrific reads.

Linear codebase history? Why even use Git then... that's SVN stuff... we use Git now-a-days for a reason...

Re: Facebook's git repo is 54GB

#66

Post by lightblade »

> @readyState would massively enjoy that first clone @feross

The first clone does not have to go over the wire. Part of git's distributed nature is that you can copy the .git to any hard drive and pass it on to someone else. Then...

> git checkout .

Re: Facebook's git repo is 54GB

#67

Post by natrius »

wtetzner wrote: post #56
indygreg2 wrote: post #17

Earlier quoted context omitted.

They aim for a completely linear history. They may even have a policy of not allowing merge commits. It is described in various places on the internet. I like https://secure.phabricator.com/book/phabflavor/article/recom... because it and its sister articles on code review and revision control are terrific reads.

Seems like you lose most of the value of git if you can't do merges.

You can still merge from master (or whatever your main development branch is) into your feature branches. You just never merge into your main branch. You squash the commits and rebase. The history ends up looking much better, and `git blame` becomes much more useful.

Re: Facebook's git repo is 54GB

#68

Post by Tobu »

maaaats wrote: post #21
danbruc wrote: post #3

Earlier quoted context omitted.

8 GB is still a lot. Would be interesting to know how much of it is actual code and how much is just images and so on.

The big .git directory is probably binary revisions? Is there any good way around that in git?

Stripping the binaries with filter-branch, or normalising them using smudge filters (for example, making sure zip-based formats use no compression).

Re: Facebook's git repo is 54GB

#69

Post by rl3 »

slig wrote: post #41

Am I missing something or this means a new intern working on a small feature, for instance, would have access to entire codebase?

It should be possible to restrict each employee's access to specific parts of the repository. However, I can't really see Facebook doing that.

Everyone having access to everything must be worth the security trade-off. On the other hand, I suppose it's debatable whether it would be a trade-off at all.

Re: Facebook's git repo is 54GB

#70

Post by vmarsy »

TD-Linux wrote: post #50
slig wrote: post #41

Am I missing something or this means a new intern working on a small feature, for instance, would have access to entire codebase?

Yes, and an intern would be subject to a code review before pushing to master - no way would they have write access to the "master" repo.

Indeed the process of facebook code deployment is pretty complex:

https://www.facebook.com/publications/514128035341603/

And I assume they have extra requirements for interns to push code.

Post reply on HN