Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement • Hacker News Board

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#31

Post by fallentimes » Thu, Aug 13, 2009, 6:22 AM UTC

videophile wrote: post #30

byrneseyeview wrote: post #27

Earlier quoted context omitted.

You're saying that someone who talks about the percentage growth in their customers would be just as concerned about the percentage shrinkage of their non-customers? Unless you're Facebook or Coca-Cola, you probably don't assume that the majority of people in the world will at some point use your product. So it doesn't make sense to measure your shrinkage of non-marketshare. The percentage change in users is the inte…

Values reported as percentages should generally be values between 0 and 100. So when the real conversion rate goes from 0.002% to 10%, the reported number should not be 499900%. I expect hackers to use metrics that are at least defined over the range they are supposed to be used. Imagine that the conversion rate goes from 0% to 1%. I report 1% improvement, the value I measured. You? (1-0)/0 = what percent?

It's _not_ a one percent improvement (1%). That is mathematically incorrect.

It's a one percentage point improvement. Wrong terminology is responsible for 95% of this thread.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#32

Post by byrneseyeview » Thu, Aug 13, 2009, 11:43 AM UTC

videophile wrote: post #30

byrneseyeview wrote: post #27

Earlier quoted context omitted.

You're saying that someone who talks about the percentage growth in their customers would be just as concerned about the percentage shrinkage of their non-customers? Unless you're Facebook or Coca-Cola, you probably don't assume that the majority of people in the world will at some point use your product. So it doesn't make sense to measure your shrinkage of non-marketshare. The percentage change in users is the inte…

Values reported as percentages should generally be values between 0 and 100. So when the real conversion rate goes from 0.002% to 10%, the reported number should not be 499900%. I expect hackers to use metrics that are at least defined over the range they are supposed to be used. Imagine that the conversion rate goes from 0% to 1%. I report 1% improvement, the value I measured. You? (1-0)/0 = what percent?

No, because that means you're using different metrics to compare the same changes, e.g. "Over the last five years, one of my stocks went up 75%. One went up 100%. One went up 125%." What purpose is served by making the last two numbers a different scale? (If you'd like to stick with the percentage terms, pretend I'm talking about dividend yield and not stock price)

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#33

Post by videophile » Thu, Aug 13, 2009, 8:11 PM UTC

byrneseyeview wrote: post #32

videophile wrote: post #30

Earlier quoted context omitted.

Values reported as percentages should generally be values between 0 and 100. So when the real conversion rate goes from 0.002% to 10%, the reported number should not be 499900%. I expect hackers to use metrics that are at least defined over the range they are supposed to be used. Imagine that the conversion rate goes from 0% to 1%. I report 1% improvement, the value I measured. You? (1-0)/0 = what percent?

No, because that means you're using different metrics to compare the same changes, e.g. "Over the last five years, one of my stocks went up 75%. One went up 100%. One went up 125%." What purpose is served by making the last two numbers a different scale? (If you'd like to stick with the percentage terms, pretend I'm talking about dividend yield and not stock price)

Bad example. The whole discussion is about reporting percentages of percentages.

Look, it's pretty clear that hackers have left the building and let marketroids rush in. Feel free to use your ill-defined metric. If you have the least bit of intellectual curiosity, you'd let everyone know what you'd report when the value goes from 0 to a non-zero value. That is all.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#34

Post by byrneseyeview » Thu, Aug 13, 2009, 8:51 PM UTC

videophile wrote: post #33

byrneseyeview wrote: post #32

Earlier quoted context omitted.

No, because that means you're using different metrics to compare the same changes, e.g. "Over the last five years, one of my stocks went up 75%. One went up 100%. One went up 125%." What purpose is served by making the last two numbers a different scale? (If you'd like to stick with the percentage terms, pretend I'm talking about dividend yield and not stock price)

Bad example. The whole discussion is about reporting percentages of percentages. Look, it's pretty clear that hackers have left the building and let marketroids rush in. Feel free to use your ill-defined metric. If you have the least bit of intellectual curiosity, you'd let everyone know what you'd report when the value goes from 0 to a non-zero value. That is all.

Yes, the whole discussion is about that. Apparently only marketroids are able to handle multiple layers of abstraction -- yes, you can take a percentage of percentages, just like you can take a percentage of dollars or a percentage of conversions.