2008 HN discussion from when the outgoing "little blue g" favicon was introduced: https://news.ycombinator.com/item?id=211518 In it, Dustin Curtis criticizes Google (rather rightly, in my hindsight-enhanced opinion) for a Marissa-Mayeresque design process of "Make 300 logo variants with pseudorandom permutations, and then pick your favorite." Looking at those permutations now ( http://2.bp.blogspot.com/_7ZYqYi4xigk/S…
The reason they did that was so that they could build up data on the effect of each factor independently. That way, when the bigwigs get into a meeting and discuss alternative designs, they can predict "Ok, this will have $X effect on revenue but will decrease search latency by Y ms, which itself will have +$X effect on revenue." I was the first engineer on the first visual redesign that tried to change everything ab…
Isn't that the problem with empirical methods in general? They are great for measuring the effect of small simple things (like small changes to a web page), but are much less effective in measuring bigger things (like say, programming language A vs. programming language B).