Google Optimize now free for everyone • Hacker News Board

Re: Google Optimize now free for everyone

#41

Post by indiesolver » Tue, Sep 04, 2018, 11:39 PM UTC

thedirt0115 wrote: post #30

indiesolver wrote: post #13

Earlier quoted context omitted.

I guess it is up to you to define your metric to be measured and to be optimized. To define the right metric and targets is not easy and probably is business-specific, i.e., is not the same for every business.

Seriously, +1 to this. Defining key metrics is something where you should sit down with a data scientist that is familiar with your domain. In my experience, the problem isn't that people can't A/B test, they don't know what to test or how to determine if an experiment was "successful".

You are right. Another detail is to find a metric which is meaningful in long term, e.g., the one which is in line with long-term happiness of your customers.

Re: Google Optimize now free for everyone

#42

Post by daveFNbuck » Wed, Sep 05, 2018, 12:39 AM UTC

RussianCow wrote: post #39

jackpirate wrote: post #28

Earlier quoted context omitted.

No one scrolls to the end of a long form article to get a recipe. Instead, they just close the tab and say "too long; didn't read." Your other point are good though.

> No one scrolls to the end of a long form article to get a recipe. I highly doubt this is true. Scrolling to the bottom of a page is a very low cost to pay for a recipe.

That was an explanation of why long articles that don't have recipes at the end aren't consistently ranked higher than short articles that don't have recipes at the end.

Re: Google Optimize now free for everyone

#43

Post by anfilt » Wed, Sep 05, 2018, 1:28 AM UTC

valarauca1 wrote: post #40

stochastic_monk wrote: post #17

Earlier quoted context omitted.

My user experience has continually depreciated on google and most websites the last few years.

Gradient Decent aims to please the average consumer, which often is not what is in-demand in the market place. Had Henry Ford used ML he would've invented a faster horse.

That mademe laugh, but true.

Re: Google Optimize now free for everyone

#44

Post by wrycoder » Wed, Sep 05, 2018, 1:41 AM UTC

stochastic_monk wrote: post #17

rightbyte wrote: post #7

"Optimizing" UI for session length and clicks is the same thing as finding the local maxima of bad, right?

My user experience has continually depreciated on google and most websites the last few years.

When I look at the all the flashy stuff Apple is planning on jamming into MacOS and iOS this fall, I expect the same will happen with my UX on those platforms. Who needs these features?

Re: Google Optimize now free for everyone

#45

Post by willio58 » Wed, Sep 05, 2018, 2:32 AM UTC

na85 wrote: post #16

matt4077 wrote: post #2

"Published Mar 30, 2017" (I'm not complaining–it's interesting and I'll give it a try. But it's not brand new)

Where is the requirement that we only see "brand-new" stuff on HN?

When the title includes “now free” but it’s old news, I think it should include the year in the title

Re: Google Optimize now free for everyone

#46

Post by RussianCow » Wed, Sep 05, 2018, 2:42 AM UTC

daveFNbuck wrote: post #42

RussianCow wrote: post #39

Earlier quoted context omitted.

> No one scrolls to the end of a long form article to get a recipe. I highly doubt this is true. Scrolling to the bottom of a page is a very low cost to pay for a recipe.

That was an explanation of why long articles that don't have recipes at the end aren't consistently ranked higher than short articles that don't have recipes at the end.

Ah, I see how I completely misread that. Thanks for clarifying!

Re: Google Optimize now free for everyone

#47

Post by jwatte » Wed, Sep 05, 2018, 2:55 AM UTC

So Google just killed crazyegg and optimizely?

Re: Google Optimize now free for everyone

#48

Post by citrablue » Wed, Sep 05, 2018, 4:37 AM UTC

roberttod wrote: post #36

citrablue wrote: post #33

Earlier quoted context omitted.

Google Optimize offers MAB testing, which is awesome. Personally, I don't see it as a "when do results become reliable" problem -- it's a "what is the cost of a false positive, false negative, or 'do nothing' decision?"

Multi Armed bandit tests need even more traffic, as there are more variations being tested. I think you have to be careful with false positives in AB testing - drawing conclusions too fast can nullify it's usefulness.

The different between "traditional" A/B testing and Multi-Armed Bandit has nothing to do with the number of treatments. A/B is really shorthand for A/B/n.

In an A/B test, the probability of selecting an "arm" (a treatment to show to the visitor) is equally distributed. Google Optimizer differs by adjusting the traffic distribution, sending more traffic to better performing options, what's usually called MAB approach.

The difference is that MABs will more quickly converge on the "winning" variation, but are more likely to get stuck in a local minimum. EG, if an "worse" option performs better right off the bat, the MAB might send most traffic to it. This would take a while for the algorithm to "recover".

The major advantages of the MAB approach are minimizing opportunity costs and the ability to capture seasonality because you can continuously run tests. Traditional split testing runs for a while, gets a result, and moves on. With a MAB, you can assign an "explore" budget that keeps tests running in the background to capture the seasonal/periodic change in conversions.

That comes with a cost: every visitor who doesn't see your best page is lost revenue.

Re: Google Optimize now free for everyone

#49

Post by dawnerd » Wed, Sep 05, 2018, 6:55 AM UTC

uberstuber wrote: post #10

rightbyte wrote: post #7

"Optimizing" UI for session length and clicks is the same thing as finding the local maxima of bad, right?

The effects of this are easiest to see with online recipes; the highest ranking recipes are all thousand word ramblings with a recipe tacked on at the end. Google sees you spent more time on the site (i.e. wasted scrolling) and thinks you were more 'engaged.'

You don't care about how that cookie recipe that only uses four ingredients changed their whole live and how their children just love it and the whole office goes nuts when they bring them in?

Yeah I don't either, I just wanna know how to make the damn cookie.

Re: Google Optimize now free for everyone

#50

Post by AstralStorm » Wed, Sep 05, 2018, 8:05 AM UTC

maxpupmax wrote: post #4

This is a freemium product. There's an enterprise upgrade that costs about 10k per year I believe. It's nice though, I use it on my site.

Additionally a freemium product where Google can steal all your ideas because you just gave them the license.

(not necessarily true, but it has happened before with YouTube)