Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling GitHub

zachholman.com

Post reply on HN
Print view View original

21–30 of 69 posts

Re: Scaling GitHub

#21

Post by akg »

I once worked at a "startup" that was the anti-culture. It hired a programmer after the current team had turned them down. That person was hired b/c she was a recommendation from of the VPs friends. At one point, we had more "Managers" in the company than people to manage...needless to say, no one got anything done.

I think most companies don't realize that businesses are always about people; they are composed of people, run by people, and sell their products/services to people. When it comes to the composition of a company, having camaraderie and talent goes hand in hand. The moment you hire someone who is average or doesn't fit, the entire work force degrades gradually.

Thomas C. Shelling developed a statistical model that actually describes this behavior and shows how a class of people clump together (http://www2.econ.iastate.edu/tesfatsi/demos/schelling/schell...). This is why talent attracts talent and mediocrity attracts mediocrity.

Re: Scaling GitHub

#22

Post by victork2 »

Pewpewarrows wrote: post #20
victork2 wrote: post #10

Earlier quoted context omitted.

Agreed, but sadly this is the state of passing information now in "new age" startup/ companies. It's like Twitter: some nice pastel colors, very little text, no information but it's easy access and people can say "Oh yeah, I know about scaling I have read this on a blog post". Sorry to break it everyone but scaling is hard, totally unfunny or uncool, with tons of problems and stressful. But to answer your question an…

Sorry to break it to you , but scaling isn't hard anymore. The tools we have at our disposal today makes it trivial compared to companies attempting to do the same thing 5-6 years ago. As soon as you figure out the scaling from 1 -> 2 servers, you're basically done until you hit top-1000 traffic. Because as soon as you spend a weekend doing that, scaling to 2 -> N is child's play. Oh look, I'll pick from any number o…

Well first I said I was in a bad mood.

More seriously, scaling is not simple. Yes products have been introduced or have evolved and simplified greatly the task. But saying that scaling is simple is simply not true. Not everybody do simple web apps, some require very heavy processing behind the scene, Hadoop/ MapReduce operations that have to return complex results in memory constrained environment and all that while maintaining low cost. As an example when Relational database are not sufficient anymore and ACID is not casting its reassuring shadow you can be in trouble if you do things like payroll/ real time process etc...

And again you don't have to be in the top 1000, many companies do not even have a lot of traffic from the general public but have to deal with very hard scaling problem.

Just look at Reddit, they are still having trouble with the Cassandra database which is said to be easiest to scale.

My 5 cents on the question!

Re: Scaling GitHub

#23

Post by alttag »

Pewpewarrows wrote: post #20
victork2 wrote: post #10

Earlier quoted context omitted.

Agreed, but sadly this is the state of passing information now in "new age" startup/ companies. It's like Twitter: some nice pastel colors, very little text, no information but it's easy access and people can say "Oh yeah, I know about scaling I have read this on a blog post". Sorry to break it everyone but scaling is hard, totally unfunny or uncool, with tons of problems and stressful. But to answer your question an…

Sorry to break it to you , but scaling isn't hard anymore. The tools we have at our disposal today makes it trivial compared to companies attempting to do the same thing 5-6 years ago. As soon as you figure out the scaling from 1 -> 2 servers, you're basically done until you hit top-1000 traffic. Because as soon as you spend a weekend doing that, scaling to 2 -> N is child's play. Oh look, I'll pick from any number o…

Okay, you've made the point that scaling technology is easy ... what is less easy (for example) is scaling the business processes to support in increased number of users or handling the communication overhead of an up-sized development team. People-centric processes cause significant scaling trouble.

Yes, aspects of scaling may be easier, but scaling (in general) is still a complex beast that kills many companies.

Re: Scaling GitHub

#24

Post by shadowfiend »

phillmv wrote: post #16
aiurtourist wrote: post #6

I'm usually wary of Holman's presentations and writing. GitHub is such a unique, special place — an engineering company with a product engineers use and are devoted to. This presentation is pretty good and has some good tenets. However, I have trouble believing that all of these ideals will work when you have a PM screaming at your engineers to finish the internal CRM system, or something else that engineers must wor…

GH is a special case. They're darlings of the tech community, and they throw a ton of events. Even if they were crappy to work for they would probably have an easy time hiring. THAT SAID, "finishing the internal CRM system" is a problem that happens in organizations an order of magnitude larger, methinks. My finishing platitude is, how did you hire - and why do you retain - engineers that don't give a shit about the…

Cause and effect? I mean, yes, they're darlings of the tech community and throw a ton of events, but that is explicitly mentioned as part of their strategy for keeping their employees happy, which is the underlying principle for their “scaling”. So perhaps it's a self-reinforcing cycle: they were originally darlings because of their service, and as they've grown, their attempts to keep their employees happy result in more goodwill, which further makes them darlings.

In essence, I think if they were crappy to work for, they wouldn't have an easy time hiring, because a lot of the cool stuff they do just wouldn't happen.

Re: Scaling GitHub

#25

Post by generalk »

aiurtourist wrote: post #6

I'm usually wary of Holman's presentations and writing. GitHub is such a unique, special place — an engineering company with a product engineers use and are devoted to. This presentation is pretty good and has some good tenets. However, I have trouble believing that all of these ideals will work when you have a PM screaming at your engineers to finish the internal CRM system, or something else that engineers must wor…

  > However, I have trouble believing that all of these 
  > ideals will work when you have a PM screaming at your 
  > engineers to finish the internal CRM system [...]
That's the point. The point of these things is that instead of having a screaming PM demanding that deadlines be met, you hire self-starting employees who have fun with their job. It's an internal CRM system, so why not shove it full of in-jokes and vi-keybindings for navigation? Why not give it a REST API and a CLI app to consume it?

It may not be possible to get an existing company to this state, but perhaps it's possible to start a fresh engineering department that's run on these principles?

Re: Scaling GitHub

#26

Post by phillmv »

shadowfiend wrote: post #24
phillmv wrote: post #16

Earlier quoted context omitted.

GH is a special case. They're darlings of the tech community, and they throw a ton of events. Even if they were crappy to work for they would probably have an easy time hiring. THAT SAID, "finishing the internal CRM system" is a problem that happens in organizations an order of magnitude larger, methinks. My finishing platitude is, how did you hire - and why do you retain - engineers that don't give a shit about the…

Cause and effect? I mean, yes, they're darlings of the tech community and throw a ton of events, but that is explicitly mentioned as part of their strategy for keeping their employees happy, which is the underlying principle for their “scaling”. So perhaps it's a self-reinforcing cycle: they were originally darlings because of their service, and as they've grown, their attempts to keep their employees happy result in…

Oh, I'm really, really open to that interpretation.

I'm currently unemployed by choice and what they describe is basically a dream job, imho.

Re: Scaling GitHub

#27

Post by generalk »

gavingmiller wrote: post #11
epenn wrote: post #8

One thing the slides mention in passing that I cannot stress the usefulness of enough is an internal wiki. There are so many common tips, issues, ideas, etc. that can be more effectively communicated and expanded upon when they're in a centralized location for all to see. So many conversations of the form "I've never dealt with that, but I know $coworker ran into that once. Go ask him." can be reduced or eliminated i…

+1 But the issue I always face is that many coworkers aren't diligent enough to document in the wiki. How would you address that?

It's gotta come from the top.

You can't dictate awesome culture like this. If the highest folks in the company update the wiki with useful information and point to it constantly, the newer hires will do the same.

If someone dictates that the wiki should be the source of information and then never puts anything in it it won't go anywhere.

Re: Scaling GitHub

#28

Post by untog »

generalk wrote: post #25
aiurtourist wrote: post #6

I'm usually wary of Holman's presentations and writing. GitHub is such a unique, special place — an engineering company with a product engineers use and are devoted to. This presentation is pretty good and has some good tenets. However, I have trouble believing that all of these ideals will work when you have a PM screaming at your engineers to finish the internal CRM system, or something else that engineers must wor…

> However, I have trouble believing that all of these > ideals will work when you have a PM screaming at your > engineers to finish the internal CRM system [...] That's the point. The point of these things is that instead of having a screaming PM demanding that deadlines be met, you hire self-starting employees who have fun with their job. It's an internal CRM system, so why not shove it full of in-jokes and vi-keybi…

perhaps it's possible to start a fresh engineering department that's run on these principles?

The key word there is department. GitHub is an engineering company, with engineers from top to bottom. It's extremely unlikely that an engineering department could live in isolation of it's company as a whole. So there would indeed be a PM screaming at someone to finish the internal CRM system and move onto something that the (non-engineer) customers really want.

Re: Scaling GitHub

#29

Post by spitfire »

phillmv wrote: post #16
aiurtourist wrote: post #6

I'm usually wary of Holman's presentations and writing. GitHub is such a unique, special place — an engineering company with a product engineers use and are devoted to. This presentation is pretty good and has some good tenets. However, I have trouble believing that all of these ideals will work when you have a PM screaming at your engineers to finish the internal CRM system, or something else that engineers must wor…

GH is a special case. They're darlings of the tech community, and they throw a ton of events. Even if they were crappy to work for they would probably have an easy time hiring. THAT SAID, "finishing the internal CRM system" is a problem that happens in organizations an order of magnitude larger, methinks. My finishing platitude is, how did you hire - and why do you retain - engineers that don't give a shit about the…

I'm not a fan of loudmouth hipster programmers either.

But it's easy to see how a programmer was hired and came to work on a project he didn't like. He was hired to work on a project he was interested in, and was moved over to a project he dreads. They won't stay long though.

Which is the difference GH has from many other places: The work is interesting (to many). Therefor they have an easy time hiring and retaining good employees.

Re: Scaling GitHub

#30

Post by spitfire »

ctide wrote: post #14
aiurtourist wrote: post #6

I'm usually wary of Holman's presentations and writing. GitHub is such a unique, special place — an engineering company with a product engineers use and are devoted to. This presentation is pretty good and has some good tenets. However, I have trouble believing that all of these ideals will work when you have a PM screaming at your engineers to finish the internal CRM system, or something else that engineers must wor…

You're right, they wouldn't. GitHub would never hire the PM you're referring to, because that's the type of PM that makes engineers hate their jobs and become demotivated.

Never say never.
Post reply on HN