I think the actual product images are more tangible to the customer compared to the illustrations. My gut feeling is that the new design and repositioning of the elements on the page don't make a huge difference aside from drawing attention to the product images.
A/B testing original design vs. random template bought from ThemeForest
21–30 of 47 posts
Re: A/B testing original design vs. random template bought from ThemeForest
#22Earlier quoted context omitted.
I used http://www.abtestcalculator.com/ and entered 3000 participants -> 28 conversions and 3000 participants -> 39 conversions. I neglected to record how many views each version had, but should be at least 3000 since the conversion ratio is about 0.5 - 1%.
The statement I see on that page is "There is a 91% chance that Variation A has a higher conversion rate". I am not sure how to interpret that. We would have to dive into the GitHub repo and figure out which test it performs I guess.
Looks like I misunderstood the result, I will change the post to reflect this.
Re: A/B testing original design vs. random template bought from ThemeForest
#23I always love this person's posts. Great informative and fun reads... Keep it up.
Re: A/B testing original design vs. random template bought from ThemeForest
#24Both pages would probably convert better if there was an explanation as to what happens when I put in my email address. There's also a confusing use over the word 'mailbox' - which I'd use for email, but also physical mail. The page asks for an email, immediately after saying "candy surprises in your mailbox" - if you assume no previous knowledge of what this site is going to offer, it's pretty confusing. And if it's…
I've always struggled to find good wording to separate physical mailboxes from electronic ones. Maybe "shipped to you twice a month" would be less vague than "in your mailbox twice a month"?
Re: A/B testing original design vs. random template bought from ThemeForest
#25I don't think this is necessarily the right conclusion. Imagine a friend with a cafe, they get the opportunity to get another cafe, same area, same clientele. With the new cafe they decide to just get some franchise, a Subway type of thing. People are familiar with it and flock there in relative droves. The original 'ma and pa' cafe is therefore deemed to be not as good as a 'template'. The other option to the franch…
How many successful businesses have you launched doing things “the right way”?
The point is there are many paths to success. Maybe candy japan didn’t do things the way you think they should be done (it’s frankly rude to refer to template hacking as cargo cult programming; we’re talking design here, not programming). But despite their “primitive approach”, they’ve been successful in multiple ways.
OP has taught me quite a bit with their experience, whereas your negative tone has taught me nothing except you believe you know how things should be done.
Honestly, you sound like a know it all.
Re: A/B testing original design vs. random template bought from ThemeForest
#26Japanese websites amaze me. The colors and density always seem so extreme.
Re: A/B testing original design vs. random template bought from ThemeForest
#27It's definitely not just the design itself. The CTA (your email submit) of the new landing page is better positioned throughout. You have also added urgency ("time until next shipment", "subscribe now before time runs out"). You can also see up to 3 previous candy boxes now, whereas before, you'd have to click through. All of those contribute to the conversion rate of your new landing page. Your old landing page actu…
This. Design != Content. Often times, simplified CTA and Value Proposition can have the same effect as a full blown redesign.
Re: A/B testing original design vs. random template bought from ThemeForest
#28Both pages would probably convert better if there was an explanation as to what happens when I put in my email address. There's also a confusing use over the word 'mailbox' - which I'd use for email, but also physical mail. The page asks for an email, immediately after saying "candy surprises in your mailbox" - if you assume no previous knowledge of what this site is going to offer, it's pretty confusing. And if it's…
I've always struggled to find good wording to separate physical mailboxes from electronic ones. Maybe "shipped to you twice a month" would be less vague than "in your mailbox twice a month"?
Perhaps "delivered to you twice a month" would be even better
Re: A/B testing original design vs. random template bought from ThemeForest
#29During this time there were a total of 67 new subscriptions. Of these 58% (39) came from the new design and 42% (28) came from the old design. Looks like the new one is a clear winner. Is it? This seems a small population to settle on a clear winner. Using R's prop test, I get a p value of 0.22. (Type "prop.test(39,67)" to calculate it). I think this means that in a world where it makes no difference which design is…
They said the following in the beginning though:
> For example if you want to test a tweak that results in 5% more conversions, you need about 3000 sales to detect it! For Candy Japan this would mean waiting for about 10 years for the test to complete.
But they want to do something still to try and improve sales. Seems reasonable even if not scientific.
Re: A/B testing original design vs. random template bought from ThemeForest
#30Both pages would probably convert better if there was an explanation as to what happens when I put in my email address. There's also a confusing use over the word 'mailbox' - which I'd use for email, but also physical mail. The page asks for an email, immediately after saying "candy surprises in your mailbox" - if you assume no previous knowledge of what this site is going to offer, it's pretty confusing. And if it's…
I've always struggled to find good wording to separate physical mailboxes from electronic ones. Maybe "shipped to you twice a month" would be less vague than "in your mailbox twice a month"?
Using the word "door" would be best I think.