Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Abba - JavaScript a/b testing

blog.alexmaccaw.com

Post reply on HN
Print view View original

11–18 of 18 posts

Re: Abba - JavaScript a/b testing

#11

Post by noelwelsh »

> If the link is navigating to an external page which you don't control, then you have no choice but to cancel the link's default event, wait a few milliseconds, then navigate manually:

Why not have a callback on the `complete` method to trigger the navigate when the request has finished?

Update for clarity: Presumably the call to the server to register a conversion uses JSONP or similar. Why not provide a callback for completion (or timeout) of this call, and use that to trigger navigation away. This seems more robust and it better encapsulated than requiring the application developer to setup a timeout.

Re: Abba - JavaScript a/b testing

#14

Post by gostevehoward »

matsiyatzy wrote: post #3

It's a bit confusing that this has the same name as another (pretty well known) a/b test calculator: http://www.thumbtack.com/labs/abba/

I'm impressed with how nicely this handles the whole A/B testing stack. But the statistics have some potentially important shortcomings -- I posted a github issue about it:

https://github.com/maccman/abba/issues/3

(Full disclosure: I authored the ABBA library mentioned by matsiyatzy, so this is incidentally self-promotional. But I wrote that tool with the hopes of improving the use and interpretation of A/B test statistics in the wider community :)

Re: Abba - JavaScript a/b testing

#15

Post by sandstrom »

karolisd wrote: post #4

Very interesting. I'll have to try it out. The first thing I'll look at is how it deals with the jQuery changes being visible to the visitors (aka flicker/flashing). Do you hide the page and wait for document to be ready? Do you do continuous checking of the DOM like Optimizely? Diagram: http://support.optimizely.com/customer/portal/articles/45288...

I agree, we use CSP and hence no inline scripts.a method to avoid flicker would be great!

Re: Abba - JavaScript a/b testing

#16

Post by ianstormtaylor »

togasystems wrote: post #10

I wonder if segment.io will be able to interface with this?

Ian from Segment.io here—randomly happened across this and I was thinking the same thing while reading it. I'd love to hear about how you were thinking it'd work, or what the abstraction would look like: ian@segment.io

Re: Abba - JavaScript a/b testing

#17

Post by ianstormtaylor »

noelwelsh wrote: post #11

> If the link is navigating to an external page which you don't control, then you have no choice but to cancel the link's default event, wait a few milliseconds, then navigate manually: Why not have a callback on the `complete` method to trigger the navigate when the request has finished? Update for clarity : Presumably the call to the server to register a conversion uses JSONP or similar. Why not provide a callback…

You don't want to force your link to depend on receiving a successful response from the server, because that would be a really bad user experience when it does break. You should opt to lose your analytics when things break instead.

To do that, you'd need to set a maximum timeout to wait for the response anyways, so then it's just a (valid) laziness call. If you _absolutely_ need the best performance then go with a success handler _and_ a maximum wait timeout. But realistically you want the maximum timeout to be ~300ms anyways, so why bother with the extra code.

Regardless, it would be nice to be able to pass a callback (whether it's based on a timeout or receiving a response) right to the `complete` method. Not sure why you're getting downvoted.

Post reply on HN