"Published Mar 30, 2017" (I'm not complaining–it's interesting and I'll give it a try. But it's not brand new)
Where is the requirement that we only see "brand-new" stuff on HN?
Hacker News :)
Google Optimize now free for everyone
21–30 of 52 posts
Re: Google Optimize now free for everyone
#22"Optimizing" UI for session length and clicks is the same thing as finding the local maxima of bad, right?
The effects of this are easiest to see with online recipes; the highest ranking recipes are all thousand word ramblings with a recipe tacked on at the end. Google sees you spent more time on the site (i.e. wasted scrolling) and thinks you were more 'engaged.'
> Google sees you spent more time on the site (i.e. wasted scrolling) and thinks you were more 'engaged.'
That seems specious. By that reasoning we could predict long form articles would triumph over short ones, but that doesn't bear out.
It seems just as plausible that there's a population of readers that do like those terrible rambling stories and tend to be more loyal to a site if they do, vs a large population who will just click on whatever recipes show up at the top of the search results with a reasonable sounding recipe name.
Recipe sites also rip each other off all the time, and it would be difficult to tell where the authoritative source of a recipe was. The personal story part is harder to rip off without being caught as a scraper (and may then boost your SEO prospects).
Re: Google Optimize now free for everyone
#23Earlier quoted context omitted.
Not necessarily - if I have a website that is trying to offer information to people about something, then it's better for everyone if it's more navigable/undersatandable, which session length / clicks might be a proxy for. That said, it certainly can be.
If your users find what they need right away, session times will go down.
If session times go down, there is a lower chance users will notice the ads, and Google wants them to see the ads.
Re: Google Optimize now free for everyone
#24I just hope they won't kill Experiments in Google Analytics.
"Google content experiments to be deprecated" https://github.com/dwyl/learn-google-optimize/issues/8
Yeah, I know, we'll use it until they actually kill it. Nothing really compelling in the Google Optimize offer make me want to switch, but maybe I am wrong.
Re: Google Optimize now free for everyone
#25Re: Google Optimize now free for everyone
#26Does anyone have a rule of thumb on when A/B testing becomes important for startups? We have a few thousand visitors a month and are starting to convert, but my guess is A/B testing language and buttons would be premature optimization for us. Just curious at what point that's no longer the case.
There are some calculators that help you figure this out. Here's one I just found with a Google search: https://www.optimizely.com/sample-size-calculator/
The idea is that to get results, you need some combination of a lot of data, or a big impact from the changes your testing. If you just change the color of the signup button, it probably won't have a major impact on the conversion rate, so you'll need a lot more data to reach a conclusion. But if you test a completely new landing page, it might have a better chance of being meaningfully different (better or worse, who knows until you test?) and so you wouldn't need as many visitors to get a result.
Re: Google Optimize now free for everyone
#27Does anyone have a rule of thumb on when A/B testing becomes important for startups? We have a few thousand visitors a month and are starting to convert, but my guess is A/B testing language and buttons would be premature optimization for us. Just curious at what point that's no longer the case.
If you have an easy way of rolling out tests, and enough data to reach statistical significance, IMHO it's never too early to test.
Re: Google Optimize now free for everyone
#28Earlier quoted context omitted.
The effects of this are easiest to see with online recipes; the highest ranking recipes are all thousand word ramblings with a recipe tacked on at the end. Google sees you spent more time on the site (i.e. wasted scrolling) and thinks you were more 'engaged.'
> Google sees you spent more time on the site (i.e. wasted scrolling) and thinks you were more 'engaged.' That seems specious. By that reasoning we could predict long form articles would triumph over short ones, but that doesn't bear out. It seems just as plausible that there's a population of readers that do like those terrible rambling stories and tend to be more loyal to a site if they do, vs a large population wh…
No one scrolls to the end of a long form article to get a recipe. Instead, they just close the tab and say "too long; didn't read."
Your other point are good though.
Re: Google Optimize now free for everyone
#29Does anyone have a rule of thumb on when A/B testing becomes important for startups? We have a few thousand visitors a month and are starting to convert, but my guess is A/B testing language and buttons would be premature optimization for us. Just curious at what point that's no longer the case.
In order to be useful, you probably want to see AB tests reaching a conclusion in under 30 days. I'd say for a conversion rate goal, this is going to be when you have around 100k visitors a month.
There are a few variables to consider
- What goal would you like to AB test? Conversion rate is an end-of-funnel goal that needs a lot of traffic, you can use upper funnel goals like product views, add to bag etc to get quicker conclusions (not as accurate but often a good approximation)
- The stats engine/AB testing tool you are using. More simple tools might conclude quicker but in my experience they can be so inaccurate they are counter productive. Usually a long time to conclude = reliable results. I've never used Google Optimize so I'm not sure where it stands.
- How many people are being exposed to the AB test, for example is it all web traffic or just mobile?
- How much of an affect the AB test has on behavior. A button color/text change will normally take long to conclude than a feature that's really helping your users.
- How confident do you want to be before reaching a conclusion? I'd recommend looking for 95% confidence in uplift before concluding an AB test.
Re: Google Optimize now free for everyone
#30"Optimizing" UI for session length and clicks is the same thing as finding the local maxima of bad, right?
I guess it is up to you to define your metric to be measured and to be optimized. To define the right metric and targets is not easy and probably is business-specific, i.e., is not the same for every business.
Seriously, +1 to this. Defining key metrics is something where you should sit down with a data scientist that is familiar with your domain. In my experience, the problem isn't that people can't A/B test, they don't know what to test or how to determine if an experiment was "successful".