There is one thing that Google has done well that no other company (Amazon, Microsoft, Facebook) seems to understand. Hire experts. They call this hire Ph.D.'s, but that's slightly inaccurate because having a Ph.D. does not make you an expert. They understand that building systems at large scale requires people who have a deep understanding of distributed systems that goes much beyond "My code is on SourceForge" mentality, or "Git is better than SVN because it is a distributed repository".
To this day, I am still shocked as to how many devs have no clue on what I'm talking about, yet they are in charge of Internet-scale systems. Here's a list of symptoms, I've heard over the years:
- I'll put something quick together.
- I implemented Paxos last night.
- I found an optimization in the two-phase commit protocol.
In my opinion, being expert means becoming humble and doubtful about your code when implementing large-scale systems. If your code runs on thousands of machines and serves 100K+ people and you think you rock as a developer/architect then you're doing something wrong.
Facebook doesn't get this. Look at their systems. They barely work. Good thing it doesn't matter. Yet. But it will eventually.
Amazon gets a little of this. Bringing Werner showed signs that they started to get it. They are still in this mix where a small group of people gets it and continues to bring in experts and push amazing things out. We'll see how long this will last.
Microsoft clearly doesn't get it. But that's ok, because they have no Internet-scale systems anyway. They built MSR which is capable of building such systems, but they make sure MSR remains isolated from their code. MSR seems happy to have no role in the company and to continue to publish amazing research.