Live data from Hacker News

Ask HN: Resources on how Google/Facebook etc. approach software design?

news.ycombinator.com

31–40 of 42 posts

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#31
I used to be at Google, years ago. I don't know if any of this is still relevant.

The design process at Google started with a design document. There was a template that I think was available online a long time ago but (ironically) I can no longer find it. The template was relatively lightweight and had some headings like so: Introduction, Goals, Non-goals, Overview, Detailed design, Security, Privacy, Testing. Compared to other design doc templates I've seen it wasn't heavy on software engineering theory. Of course the bulk of the writing would be in the "detailed design" section and subsections.

Good design docs were very detailed. One I wrote ended up being, I think, about 40 pages by the end when printed out, and that was not an especially large or unusual document. Design docs for critical systems could be larger still, or more frequently, split into many other docs. The quality of writing was generally high and they were maintained in version control. There was a mailing list where design docs were posted for company-wide review, though by the time I was there, this process had degraded quite a bit and a lot of stuff was done in team-specific design docs in Google Docs, with relatively minimal or no peer review. I felt that it was common for the less "serious" teams to do this, e.g. teams working on the latest chat product or on Google Apps itself. Those docs tended to be shorter, only partly filled out, or non-existent. The closer to the metal, older-school stuff was all hand-written HTML.

Good design docs would be kept up to date as the design evolved, although I'd say that was the minority. Most designs were the work of a small number of people or just one person. There were not many design review meetings that I recall, though probably that varied a lot by team.

Diagrams were minimal, possibly because there weren't any good diagramming tools available internally (well, there was graphviz and TeX).

Data structures would often be designed up front as long as they were either protocol buffers (i.e. inter-server comms or long term data storage), or fundamental to what the system did as with BigTable, indexing, index serving, ad serving etc. Systems where the data structures weren't fundamental to the design didn't necessarily plan out every structure in advance of course, by no means. For many products the user interface or network protocols were more important, so that's where the design docs would dwell.

The most senior engineers were very familiar with the performance costs of things, e.g. the cost of an L2 cache miss vs a disk seek, and that deeply informed the design of many systems.

That's about it.

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#33
Ex Amazon here. Amazon has some solid principles that often go against popular belief.

- No waterfall-ish process where design is handed down from architects to senior engineers to juniors. The same people do design, implementation, ops and so on.

- Measure everything and always. People are encouraged to define metrics and goals and create dashboards before writing code

- Simplify: decreasing complexity is taken more seriously than in other companies. Do not use a database when you can use a file, or a message passing library when you can use a socket, or 200 lines of code when you can for out to "grep | sort". This can be surprising to new hires.

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#34
post #20

Earlier quoted context omitted.

What are some critical parts of Facebook written in PHP, if you know?

The entire web fronted, and most site “business logic” is implemented in Hack (sort of PHP, statically typed)

"Hack" correct name

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#35
post #30
post #24

Earlier quoted context omitted.

This is probably true for Google, but not Facebook, given how many times they have broken things, especially security related.

Facebook's culture is basically a younger brother of Google's, or at least used to be. There's a reason their tech stacks are so similar - FB went through a phase where it poached a lot of Googlers. A tech firm is most easily defined as a company founded and run by a software or hardware engineer. The only exception I can think of is Apple, but Jobs was immersed in engineering culture from the time he was a teenager…

"their tech stacks are so similar" in what ways? not similar at all

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#36

Ex Amazon here. Amazon has some solid principles that often go against popular belief. - No waterfall-ish process where design is handed down from architects to senior engineers to juniors. The same people do design, implementation, ops and so on. - Measure everything and always. People are encouraged to define metrics and goals and create dashboards before writing code - Simplify: decreasing complexity is taken more…

> message passing library when you can use a socket

kindly explain that logic

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#37
post #30

Earlier quoted context omitted.

Facebook's culture is basically a younger brother of Google's, or at least used to be. There's a reason their tech stacks are so similar - FB went through a phase where it poached a lot of Googlers. A tech firm is most easily defined as a company founded and run by a software or hardware engineer. The only exception I can think of is Apple, but Jobs was immersed in engineering culture from the time he was a teenager…

"their tech stacks are so similar" in what ways? not similar at all

Thrift RPC is basically a clone of Stubby. Lots of sharded MySQL (not so much at Google anymore but the whole time I was there, the ads db was a giant sharded MySQL DB). Small number of large monorepos. Etc.

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#38
post #36

Ex Amazon here. Amazon has some solid principles that often go against popular belief. - No waterfall-ish process where design is handed down from architects to senior engineers to juniors. The same people do design, implementation, ops and so on. - Measure everything and always. People are encouraged to define metrics and goals and create dashboards before writing code - Simplify: decreasing complexity is taken more…

> message passing library when you can use a socket kindly explain that logic

I'm also curious about this. Reinventing your own serialization and retransmission every time, instead of using proven technology does not sound like "decreasing" complexity. Same goes for using files and grep instead of databases in many cases.

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#39
post #22
post #6

Earlier quoted context omitted.

In my experience working at Facebook, not a single thing you said was true. People plan and think before they code. People are rewarded for impact, not for landing useless code. Code is carefully designed and tested, or it doesn't land. Impact rules all, not velocity.

If these principles are followed throughout the engineering org, can you explain how the recent reports of password and other information being stored in plaintext happened? That is Security 101. Genuinely curious.

No idea haven't worked at Facebook for over four years

Re: Ask HN: Resources on how Google/Facebook etc. approach software design?

#40
post #38
post #36

Earlier quoted context omitted.

> message passing library when you can use a socket kindly explain that logic

I'm also curious about this. Reinventing your own serialization and retransmission every time, instead of using proven technology does not sound like "decreasing" complexity. Same goes for using files and grep instead of databases in many cases.

It probably means that if you only need to ping a process from another one (no payload needed or only a handful of case expeceted) you don't have to deploy your last gRPC knowledge. cf. how daemons are managed on Linux
Post reply on HN