Note on SumAll All users who use SumAll should be wary of their service. We tried them out and we then found out that they used our social media accounts to spam our followers and users with their advertising. We contacted them asking for answers and we never heard from them. Our suggestion: Avoid SumAll.
Hey Antr, Jacob from SumAll here. Sorry to hear you had a bad experience with us. The tweets you're talking about that "spam" your accounts were most likely the performance tweets that you are free to toggle on and off. Here's how you can do that: https://support.sumall.com/customer/portal/articles/1378662-... Best, Jacob
Winning A/B results were not translating into improved user acquisition
11–20 of 67 posts
Re: Winning A/B results were not translating into improved user acquisition
#12Here's the thing, stop A/Bing every little thing (and/or "just because") and you'll get more significant results.
Do you think the true success of something is due to A/B testing? A/B testing is optimizing, not archtecting.
Re: Winning A/B results were not translating into improved user acquisition
#13Note on SumAll All users who use SumAll should be wary of their service. We tried them out and we then found out that they used our social media accounts to spam our followers and users with their advertising. We contacted them asking for answers and we never heard from them. Our suggestion: Avoid SumAll.
Hey Antr, Jacob from SumAll here. Sorry to hear you had a bad experience with us. The tweets you're talking about that "spam" your accounts were most likely the performance tweets that you are free to toggle on and off. Here's how you can do that: https://support.sumall.com/customer/portal/articles/1378662-... Best, Jacob
Re: Winning A/B results were not translating into improved user acquisition
#14Re: Winning A/B results were not translating into improved user acquisition
#15This article comes off as a bit boastful and somewhat of an advertisement for the company... "What threw a wrench into the works was that SumAll isn’t your typical company. We’re a group of incredibly technical people, with many data analysts and statisticians on staff. We have to be, as our company specializes in aggregating and analyzing business data. Flashy, impressive numbers aren’t enough to convince us that th…
Re: Winning A/B results were not translating into improved user acquisition
#16Re: Winning A/B results were not translating into improved user acquisition
#17this is all fine and good, but if you're goal is to see what works best between X new versions of a page and you are rigorous in creating variants, Optimizely is a great tool for figuring out the best converting variant.
Re: Winning A/B results were not translating into improved user acquisition
#18I was also disturbed that the effect size was taken into account in the sample size selection. You need to know this before you do any type of statistical test. Otherwise, you are likely to get "positive" results that just don't mean anything.
OTOH, I wasn't too concerned that the test was a one-tailed test. Honestly, in a website A/B test, all I really am concerned about is whether my new page is better than the old page. A one-tailed test tells you that. It might be interesting to run two-tailed tests just so you can get an idea what not to do, but for this use I think a one-tailed test is fine. It's not like you're testing drugs, where finding any effect, either positive or negative, can be valuable.
I should also note that I only really know enough about statistics to not shoot myself in the foot in a big, obvious way. You should get a real stats person to work on this stuff if your livelihood depends on it.
Re: Winning A/B results were not translating into improved user acquisition
#19Optimizely actually has a decent article on it: https://help.optimizely.com/hc/en-us/articles/200040355-Run-...
Re: Winning A/B results were not translating into improved user acquisition
#20Note on SumAll All users who use SumAll should be wary of their service. We tried them out and we then found out that they used our social media accounts to spam our followers and users with their advertising. We contacted them asking for answers and we never heard from them. Our suggestion: Avoid SumAll.
Hey Antr, Jacob from SumAll here. Sorry to hear you had a bad experience with us. The tweets you're talking about that "spam" your accounts were most likely the performance tweets that you are free to toggle on and off. Here's how you can do that: https://support.sumall.com/customer/portal/articles/1378662-... Best, Jacob