Live data from Hacker News

Leaked UI A/B Tests from Major Websites

goodui.org

11–17 of 17 posts

Re: Leaked UI A/B Tests from Major Websites

#11
post #4

I don’t think that in every case it is necessary down to user rejections. I can’t believe that in the netflix case users actually prefer to have to login with two post backs, one for the login, one for the password.

People are probably not registering right away that they are signing up for an account.

Re: Leaked UI A/B Tests from Major Websites

#12
post #4

I don’t think that in every case it is necessary down to user rejections. I can’t believe that in the netflix case users actually prefer to have to login with two post backs, one for the login, one for the password.

This is a necessary pattern for increased security.

What’s the security difference with two instead of one?

Re: Leaked UI A/B Tests from Major Websites

#14
post #4

I don’t think that in every case it is necessary down to user rejections. I can’t believe that in the netflix case users actually prefer to have to login with two post backs, one for the login, one for the password.

It wouldn't surprise me if people with active netflix accounts tried to login into that page which is why they changed how it works, also like with any A/B testing it's not always clear what user behaviour they were seeking to change or reinforce, and it's usually more than one.

For example the buy now vs add to cart only on the Amazon one might have been looking at more than just how many products are sold, they might were also been trying to see if they can say reduce impulse buys that result in returns without lowering purchases that do not, in fact the reason they've kept the buy now might be because it actually reduced the return rate as people interacted less with the site and didn't buy additional items that they returned later.

Re: Leaked UI A/B Tests from Major Websites

#15

In many of these cases, the "A/B test" may have been accidental. Running a software rollout is frequently done slowly, datacenter by datacenter, and during that time some people might see one version and others might see another. From the users perspective it looks the same as an A/B test, but the difference is nobody was looking at the results...

Almost any user-noticable change (that is not a bugfix) is run as a AB-test for a few weeks in my team to verify that it does have the intended impact. I'd be surprised if there aren't teams at google, amazon, netflix and similar organizations that work similarly.

Re: Leaked UI A/B Tests from Major Websites

#16
post #13

"Leaked..." How are these leaked? Did someone hack into something?

They probably mean "detected", as in users noticed something behaving differently between devices or after clearing cookies.

It could also be looking for possible A/B flags in cookies/localStorage.

Re: Leaked UI A/B Tests from Major Websites

#17

Earlier quoted context omitted.

This is a necessary pattern for increased security.

What’s the security difference with two instead of one?

You can have more sophisticated algorithms like SRP that don't send password hashes.

https://en.wikipedia.org/wiki/Secure_Remote_Password_protoco...

Post reply on HN