One thing the slides mention in passing that I cannot stress the usefulness of enough is an internal wiki. There are so many common tips, issues, ideas, etc. that can be more effectively communicated and expanded upon when they're in a centralized location for all to see. So many conversations of the form "I've never dealt with that, but I know $coworker ran into that once. Go ask him." can be reduced or eliminated i…
Scaling GitHub
11–20 of 69 posts
Re: Scaling GitHub
#12One thing the slides mention in passing that I cannot stress the usefulness of enough is an internal wiki. There are so many common tips, issues, ideas, etc. that can be more effectively communicated and expanded upon when they're in a centralized location for all to see. So many conversations of the form "I've never dealt with that, but I know $coworker ran into that once. Go ask him." can be reduced or eliminated i…
+1 But the issue I always face is that many coworkers aren't diligent enough to document in the wiki. How would you address that?
Re: Scaling GitHub
#13Totally devoid of information. I would be more interested in reading actual technical reports of higly scaled infrastructures. These flimsy half-truths gleaned from such scaling, I can do without.
Agreed, but sadly this is the state of passing information now in "new age" startup/ companies. It's like Twitter: some nice pastel colors, very little text, no information but it's easy access and people can say "Oh yeah, I know about scaling I have read this on a blog post". Sorry to break it everyone but scaling is hard, totally unfunny or uncool, with tons of problems and stressful. But to answer your question an…
2) Scaling is also… really custom. I'm extremely unlikely to have to find a way to shard git repositories across a network of users. And they do cover their architecture and release back a fair number of their tools.
Re: Scaling GitHub
#14I'm usually wary of Holman's presentations and writing. GitHub is such a unique, special place — an engineering company with a product engineers use and are devoted to. This presentation is pretty good and has some good tenets. However, I have trouble believing that all of these ideals will work when you have a PM screaming at your engineers to finish the internal CRM system, or something else that engineers must wor…
Re: Scaling GitHub
#15Totally devoid of information. I would be more interested in reading actual technical reports of higly scaled infrastructures. These flimsy half-truths gleaned from such scaling, I can do without.
That seems a little harsh for something that is clearly intended to be an overview of general principles, not specific technical details.
Re: Scaling GitHub
#16I'm usually wary of Holman's presentations and writing. GitHub is such a unique, special place — an engineering company with a product engineers use and are devoted to. This presentation is pretty good and has some good tenets. However, I have trouble believing that all of these ideals will work when you have a PM screaming at your engineers to finish the internal CRM system, or something else that engineers must wor…
Even if they were crappy to work for they would probably have an easy time hiring.
THAT SAID, "finishing the internal CRM system" is a problem that happens in organizations an order of magnitude larger, methinks.
My finishing platitude is, how did you hire - and why do you retain - engineers that don't give a shit about the project they're working on?
Also, why is your PM screaming? A project meeting deadlines is his or her responsibility. Sounds like you have a crap PM to boot.
Re: Scaling GitHub
#17Totally devoid of information. I would be more interested in reading actual technical reports of higly scaled infrastructures. These flimsy half-truths gleaned from such scaling, I can do without.
Re: Scaling GitHub
#18What graphing utility/service do you guys use at GitHub?
Re: Scaling GitHub
#19Totally devoid of information. I would be more interested in reading actual technical reports of higly scaled infrastructures. These flimsy half-truths gleaned from such scaling, I can do without.
Agreed, but sadly this is the state of passing information now in "new age" startup/ companies. It's like Twitter: some nice pastel colors, very little text, no information but it's easy access and people can say "Oh yeah, I know about scaling I have read this on a blog post". Sorry to break it everyone but scaling is hard, totally unfunny or uncool, with tons of problems and stressful. But to answer your question an…
Re: Scaling GitHub
#20Totally devoid of information. I would be more interested in reading actual technical reports of higly scaled infrastructures. These flimsy half-truths gleaned from such scaling, I can do without.
Agreed, but sadly this is the state of passing information now in "new age" startup/ companies. It's like Twitter: some nice pastel colors, very little text, no information but it's easy access and people can say "Oh yeah, I know about scaling I have read this on a blog post". Sorry to break it everyone but scaling is hard, totally unfunny or uncool, with tons of problems and stressful. But to answer your question an…
Oh look, I'll pick from any number of battle-tested, high performance, and free load balancers. I'll spin up a couple more VPS nodes. I'll add a few more slaves to my DB setup. I'll install some memcached and a quick write-through wrapper in my code. Oh yeah, duh, don't do long-running operations on the request/response cycle, here's an open source queue system for you. And since I like going overboard, let's throw some Varnish all over the place. Congrats, we've just covered the scaling concerns of 95% of web apps.
Try doing that several years ago with the terrible PHP and Java services we were writing, and without most of the hardened tools we now get from thirty seconds of Googling and learning from everyone else's mistakes.
Disclaimer: yes, when you start to break through the top-1000 some hard problems start appearing again. You begin to run into the limits of what others have created, and patching / rolling your own solutions becomes commonplace.