Live data from Hacker News

Optimizely Statistics Engine

optimizely.com

11–20 of 57 posts

Re: Optimizely Statistics Engine

#11
post #6

Subject-verb agreement for the internet age will have to wait for a while.

Perhaps the word "statistics" is singular, like "mathematics".

It's something I've heard both ways. Whether it's correct in both cases ... I'm not sure. "I end up feeling like a statistic" is a singular form, for sure, but I often hear/read "statistics" referenced singularly as well.

Re: Optimizely Statistics Engine

#12
post #4

Earlier quoted context omitted.

"Statistics reinvented for the internet age" looks like a cheap knock-off of the long known bayesian statistics.

It's actually sequential testing which is not necessarily Bayesian. Also, it's not a cheap knock off. Just another way of getting to an answer.

Bayesian updating of the posterior? Or if you prefer frequentist algorithms for online learning of classifiers?

Not trying to pick a fight, just as a statistitian/ML developer I've seen the same things be reinvented and renamed so many times.

Re: Optimizely Statistics Engine

#15
post #12

Earlier quoted context omitted.

It's actually sequential testing which is not necessarily Bayesian. Also, it's not a cheap knock off. Just another way of getting to an answer.

Bayesian updating of the posterior? Or if you prefer frequentist algorithms for online learning of classifiers? Not trying to pick a fight, just as a statistitian/ML developer I've seen the same things be reinvented and renamed so many times.

No, if you read their technical paper, it's frequentist sequential testing with false discovery rate control, which is a fairly recent development (I mean, 25 years old is pretty new in statistics).

http://pages.optimizely.com/rs/optimizely/images/stats_engin...

Re: Optimizely Statistics Engine

#17

Earlier quoted context omitted.

Perhaps the word "statistics" is singular, like "mathematics".

It's something I've heard both ways. Whether it's correct in both cases ... I'm not sure. "I end up feeling like a statistic" is a singular form, for sure, but I often hear/read "statistics" referenced singularly as well.

Statistics refers to the subject (usually), while a statistic has a specific meaning within statistics.

Re: Optimizely Statistics Engine

#18
post #9

This graph is a joke: https://d1qmdf3vop2l07.cloudfront.net/optimizely-marketer-as... And any company trying to sell a statistical tool/package that would actually create a graph like that is selling snake oil. Your model only gets better, digitally, and never sees a regression? And you're using this for web analytics?

I don't think the graph is particularly good either, but I think you're maybe reading it wrong too. The y axis is significance level which makes sense would normally improve as visitors go up. Their line arguably is a moving average regression.

Re: Optimizely Statistics Engine

#19
post #18
post #9

This graph is a joke: https://d1qmdf3vop2l07.cloudfront.net/optimizely-marketer-as... And any company trying to sell a statistical tool/package that would actually create a graph like that is selling snake oil. Your model only gets better, digitally, and never sees a regression? And you're using this for web analytics?

I don't think the graph is particularly good either, but I think you're maybe reading it wrong too. The y axis is significance level which makes sense would normally improve as visitors go up. Their line arguably is a moving average regression.

He's saying that as new data arrives, it has to adjust both ways, not only toward the correct answer.

(If that wasn't the case, you could just figure out which was the only direction it would move and then stop collecting data. You've already got your answer)

Re: Optimizely Statistics Engine

#20
I am surprised by all the negative commentary here. On the whole, companies like Optimizely, RJMetrics, Custora, and others are doing more to push statistical analysis to the mass market than anyone else. These tools are not designed for statisticians or ML practitioners so it makes sense they do not put language like Bayesian, etc. front and center. IMO, the more people using data to make decisions, the better.
Post reply on HN