Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement • Hacker News Board

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#11

Post by [deleted] » Wed, Aug 12, 2009, 5:17 PM UTC

rbanffy wrote: post #10

Hi folks. I am conducting an experiment (this is phase 2) and need about 50 downvotes. Would you please downvote this post?

[deleted]

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#12

Post by minsight » Wed, Aug 12, 2009, 5:22 PM UTC

rbanffy wrote: post #10

Hi folks. I am conducting an experiment (this is phase 2) and need about 50 downvotes. Would you please downvote this post?

There is a time and a place for experiments. I'm pretty sure it's not here and not now.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#13

Post by rbanffy » Wed, Aug 12, 2009, 5:23 PM UTC

[deleted] wrote: post #11

rbanffy wrote: post #10

Hi folks. I am conducting an experiment (this is phase 2) and need about 50 downvotes. Would you please downvote this post?

[deleted]

Wondering how much karma would I lose if I were flagged... Please, go ahead.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#14

Post by rbanffy » Wed, Aug 12, 2009, 5:25 PM UTC

[deleted] wrote: post #11

rbanffy wrote: post #10

Hi folks. I am conducting an experiment (this is phase 2) and need about 50 downvotes. Would you please downvote this post?

[deleted]

And if I were a comment spammer, I would probably embed a link or two in my message ;-)

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#15

Post by rbanffy » Wed, Aug 12, 2009, 5:28 PM UTC

minsight wrote: post #12

rbanffy wrote: post #10

Hi folks. I am conducting an experiment (this is phase 2) and need about 50 downvotes. Would you please downvote this post?

There is a time and a place for experiments. I'm pretty sure it's not here and not now.

If an experiment has HN as its subject, this is precisely not the right but the only place it can happen.

As for the time, well... I got 40+ upvotes today (I started this morning needing about half a dozen).

The plan is to reach 1337 karma points (did it yesterday, but I wasn't monitoring to get a screenshot).

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#16

Post by ramit » Wed, Aug 12, 2009, 5:31 PM UTC

onreact-com wrote: post #3

"Conclusion: Simplified design improved conversion by an average of 20.45%." is the most important sentence in this post. To be honest you don't have to use Google Website Optimizer to notice that.

Yeah you do. I ran a test on a friend's site, convinced that a short version would outperform the long-copy version. Conversions dropped by about 30%, which would have cost him tens of thousands of dollars per year.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#17

Post by minsight » Wed, Aug 12, 2009, 5:31 PM UTC

rbanffy wrote: post #15

minsight wrote: post #12

Earlier quoted context omitted.

There is a time and a place for experiments. I'm pretty sure it's not here and not now.

If an experiment has HN as its subject, this is precisely not the right but the only place it can happen. As for the time, well... I got 40+ upvotes today (I started this morning needing about half a dozen). The plan is to reach 1337 karma points (did it yesterday, but I wasn't monitoring to get a screenshot).

Then that's not an experiment, but a spammy attempt at satisfying your vanity. This is not what HN is for.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#18

Post by webwright » Wed, Aug 12, 2009, 5:35 PM UTC

videophile wrote: post #7

jbm wrote: post #5

Earlier quoted context omitted.

It's been a while since I've taken a math class, but I'm not sure what the problem is with calling it a 21.2% improvement. The old conversion rate was 24.4%. The new one was 29.6%. (29.6 - 24.4) / 24.4 = .213 = 21.3%. Is it misleading because their number of conversions is so low? (Not trying to be snarky, I have no idea)

The improvement in the conversion rate is (29.6 - 24.4) = 5.2%. Reporting "a percent of a percent" is downright misleading because it is entirely dependent on the starting baseline and also because conversion rates have high natural variance. Imagine an improvement from 2% conversion (for a truly awful site) to 2.6% (for a site just as awful). That "30% improvement" just isn't. If they were intellectually honest and…

Huh.

The number that matters is "how many more users are signing up since we changed"? The answer is NOT 5.2%.

Example: I have a 1% conversion rate on 100,000 visitors. 1,000 customers! Yay! I change something and I get it up to 2% on the next 100,000 visitors. 2,000 customers.

Which is a better description: "I doubled my conversion rate" or "I increased my conversion rate by 1%?" I'd go with the former.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#19

Post by rbanffy » Wed, Aug 12, 2009, 5:37 PM UTC

minsight wrote: post #17

rbanffy wrote: post #15

Earlier quoted context omitted.

If an experiment has HN as its subject, this is precisely not the right but the only place it can happen. As for the time, well... I got 40+ upvotes today (I started this morning needing about half a dozen). The plan is to reach 1337 karma points (did it yesterday, but I wasn't monitoring to get a screenshot).

Then that's not an experiment, but a spammy attempt at satisfying your vanity. This is not what HN is for.

Actually, I am trying to hack the meatware that operates on HN. That's worthy of hacker news, isn't it?

As long as I am kept downvoted, my posts will be a nice grey and way down the list. Very little harm comes from that.

Re: Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement

#20

Post by rbanffy2 » Wed, Aug 12, 2009, 5:46 PM UTC

rbanffy wrote: post #19

minsight wrote: post #17

Earlier quoted context omitted.

Then that's not an experiment, but a spammy attempt at satisfying your vanity. This is not what HN is for.

Actually, I am trying to hack the meatware that operates on HN. That's worthy of hacker news, isn't it? As long as I am kept downvoted, my posts will be a nice grey and way down the list. Very little harm comes from that.