How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
1–10 of 17 posts
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#2The devil's in the details, but the angels are there, too.
Amazing how much impact these "minor" changes can have on the bottom line.
Thanks for the reminder to split test all things great and small.
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#3The devil's in the details, but the angels are there, too. Amazing how much impact these "minor" changes can have on the bottom line. Thanks for the reminder to split test all things great and small.
It's funny how that works. I suspect social proof was a distraction on that optin form.
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#4I wish I knew more about statistics. Punching just the two variants that made it to the end into a Chi square calculator gave me a 90% confidence, not the 95% he cited.
I also wonder about stopping the variation early, and even the conditions under which he stopped the test. http://www.evanmiller.org/how-not-to-run-an-ab-test.html talks about pitfalls when "testing until you reach confidence".
Are there any better references for these types of questions?
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#5I wish I knew more about statistics. Punching just the two variants that made it to the end into a Chi square calculator gave me a 90% confidence, not the 95% he cited. I also wonder about stopping the variation early, and even the conditions under which he stopped the test. http://www.evanmiller.org/how-not-to-run-an-ab-test.html talks about pitfalls when "testing until you reach confidence". Are there any better re…
I cited 95% confidence, which was calculated by Visual Website Optimizer. I wonder how they calculate their confidence level.
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#6Not only is it a distraction. I see that and think "You are going to send me email that will be sent to 15,000 other people? That would just be spam for me! Why should I sign up if you are going to spam me?"
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#7Not sure if it's significant yet. What he really got was: 9 people click on the default form, 5 people on variation one, and 14 people on variation two. These are very low numbers no matter what way you look at them.
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#8I wish I knew more about statistics. Punching just the two variants that made it to the end into a Chi square calculator gave me a 90% confidence, not the 95% he cited. I also wonder about stopping the variation early, and even the conditions under which he stopped the test. http://www.evanmiller.org/how-not-to-run-an-ab-test.html talks about pitfalls when "testing until you reach confidence". Are there any better re…
I cited 95% confidence, which was calculated by Visual Website Optimizer. I wonder how they calculate their confidence level.
Never used Visual Website Optimizer, but chances are, whatever they do leaves you vulnerable to the flaw mentioned in the article that swalberg linked.
Basically, every time you peek at the results of a split test to see if the confidence level is high enough, you make your split test more prone to error.
There are ways to adjust for this which allow you to continuously monitor the test and stop if it hits a threshold, but it's much easier just to set a known number of samples to take and accept the confidence level that comes out then.
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#9I'm wondering if merely displaying a number is really "social proof" in the Cialdinian sense.
Possibly at one time, but now I think the "proof" part of social proof must be demonstrably not BS for it to work. In the same way that testimonials by "J.S., Ohio" have no real meaning anymore, but "Jared Sanderson, CEO of Whatever.com" would probably be taken seriously.
An interesting experiment would be to get permission from some of your better-known users to say something like "Join Bill Gates, Dalai Lama, and 14,152 others...". Possibly even better to use little avatars or pics of a bunch of people, ala Facebook Like or the old BlogLog widget. Who knows? I feel like that'd be a better thumbs up/down on social proof than just the number.
Re: How Split-Testing Our Opt-In Form Increased Our Conversion Rate by 102.2%
#10I wish I knew more about statistics. Punching just the two variants that made it to the end into a Chi square calculator gave me a 90% confidence, not the 95% he cited. I also wonder about stopping the variation early, and even the conditions under which he stopped the test. http://www.evanmiller.org/how-not-to-run-an-ab-test.html talks about pitfalls when "testing until you reach confidence". Are there any better re…
Founder of Visual Website Optimizer here.
Couple of articles you can refer on math we use:
http://visualwebsiteoptimizer.com/split-testing-blog/what-yo...
http://visualwebsiteoptimizer.com/split-testing-blog/ab-test...