Ask HN: Has anyone read the “Focusing on the Long-term” paper from Google? [pdf]

#1

Post by root_node » Mon, Aug 10, 2020, 7:27 PM UTC

Ask HN: Has anyone read the “Focusing on the Long-term” paper from Google? [pdf]
storage.googleapis.com

Re: Ask HN: Has anyone read the “Focusing on the Long-term” paper from Google? [pdf]

#2

Post by root_node » Mon, Aug 10, 2020, 7:27 PM UTC

Almost every company that's out there runs some form of A/B tests to measure the success/failure of their features. Typically, A/B tests span a few weeks (2-3). Is that timeframe enough to make critical decisions? What's your take on short lived A/B tests? I found the paper interesting. Has anyone tried this/something similar outside of Google?

Re: Ask HN: Has anyone read the “Focusing on the Long-term” paper from Google? [pdf]

#3

Post by cloudking » Mon, Aug 10, 2020, 9:13 PM UTC

root_node wrote: post #2

Almost every company that's out there runs some form of A/B tests to measure the success/failure of their features. Typically, A/B tests span a few weeks (2-3). Is that timeframe enough to make critical decisions? What's your take on short lived A/B tests? I found the paper interesting. Has anyone tried this/something similar outside of Google?

I think A/B testing largely depends on your sample sizes. If you have a very high traffic product, it's easier to get statistical significance in a shorter time frame.