Live data from Hacker News

Counting Objects

githubengineering.com

11–20 of 40 posts

Re: Counting Objects

#11
post #8

This is awesome. I absolutely love engineering blogs like this. There's no real reason for companies to educate external devs/hobbyists/students like this, but some do, and it's really awesome.

The reason is recruiting, and it's effective.

Because people read it and think "hey that sounds cool, I want to work there"?

Re: Counting Objects

#12
post #11

Earlier quoted context omitted.

The reason is recruiting, and it's effective.

Because people read it and think "hey that sounds cool, I want to work there"?

Pretty much.

There are second order effects as well. Having a good tech blog that talks about this stuff gains the company mindshare amongst developers. If, as in in the case of github, the developers (etc) are the customer, then the translation to marketshare is pretty straight-forward.

When developers (etc) are not the target audience, they still get warm-fuzzies about the company and it gains word of mouth, and non-tech who seek out a recommendation from their friends, will hear about the company with the good blog. Usually in the form of "Oh yeah, there are a couple of alternatives, but the tech at $X is top notch, look hard at them".

Re: Counting Objects

#13
post #11

Earlier quoted context omitted.

Because people read it and think "hey that sounds cool, I want to work there"?

Pretty much. There are second order effects as well. Having a good tech blog that talks about this stuff gains the company mindshare amongst developers. If, as in in the case of github, the developers (etc) are the customer, then the translation to marketshare is pretty straight-forward. When developers (etc) are not the target audience, they still get warm-fuzzies about the company and it gains word of mouth, and no…

Hm yeah, that makes sense. So it's not the case that "[there are] no real reasons" - but I still think it's awesome, and learn a lot/see the applications of stuff learned elsewhere in them.

Re: Counting Objects

#14
This is technical recruiting done right, a lucid walkthrough of a hard problem complete with links to the implementation. It must have taken at least a few weeks of engineer-time to write, Github is awesome for making this public.

I wish they had talked a little more about the tradeoff they made. They mentioned that splitting packfiles by fork was space-prohibitive, but ended up with a solution which must take more space than originally used. (If the new heuristic refuses to use some objects as delta bases, some options which would have provided the best compression are no longer available to git)

The performance win is incredible, how much space did they give up in the process?

Re: Counting Objects

#15
post #6
post #5

Earlier quoted context omitted.

Yes, much of GitHub's core infrastructure is written in C.

That's awesome. You don't find many people that enjoy writing in C. I assumed it was Ruby.

Ruby MRI and YARV are both written in C, so to be a true Ruby hacker you need some C chops. :-)

Re: Counting Objects

#16

This is technical recruiting done right, a lucid walkthrough of a hard problem complete with links to the implementation. It must have taken at least a few weeks of engineer-time to write, Github is awesome for making this public. I wish they had talked a little more about the tradeoff they made. They mentioned that splitting packfiles by fork was space-prohibitive, but ended up with a solution which must take more s…

The fork-aware packfiles are bigger, but not as much as you might think. We give up the "best" delta that git can find, but usually there is another one in the same group that is almost as good.

I didn't have any numbers on hand, so I just repacked our git/git network.git with and without the delta-aware code. A stock repack is about 300MB, and the delta-aware one is 400MB.

That sounds like a lot, but the real alternative is not sharing objects at all, which is more like 350GB.

Re: Counting Objects

#17
post #11

Earlier quoted context omitted.

The reason is recruiting, and it's effective.

Because people read it and think "hey that sounds cool, I want to work there"?

Or, perhaps even more importantly, “they really seem to know what they're doing. I'm going to put my company's projects there”.

If you sell developer tools, this is _way_ more effective than an ad campaign because it's completely genuine.

Re: Counting Objects

#18

This is technical recruiting done right, a lucid walkthrough of a hard problem complete with links to the implementation. It must have taken at least a few weeks of engineer-time to write, Github is awesome for making this public. I wish they had talked a little more about the tradeoff they made. They mentioned that splitting packfiles by fork was space-prohibitive, but ended up with a solution which must take more s…

This is an actual interesting problem that takes skill and knowledge to solve and it actually produces immense business value too (at least for Github). This is the kind of stuff more software engineers should be working on. I'm dying over here working on a small React JS component and some CSS layout tinkering, my brain's got graph traversal algorithms and bitmap indexes on its mind and can't do anything about it.

Re: Counting Objects

#19
post #6

Earlier quoted context omitted.

That's awesome. You don't find many people that enjoy writing in C. I assumed it was Ruby.

Ruby MRI and YARV are both written in C, so to be a true Ruby hacker you need some C chops. :-)

Interesting. I've heard that thrown around before, but I've never heard someone say the same thing about Python, even though the main Python implementation is C.

Re: Counting Objects

#20
I wonder why GitHub has a separate domain githubengineering.com for this blog instead of a subdomain like engineering.github.com.

I notice that there is an inactive user account called engineering. If at all an User Page is created by that account, it would be available at engineering.github.io.

Post reply on HN