A/B testing gets misused to juice metrics in the short term

Without a metric to say what is "better" and a method to measure it this is empty advice.

    1. Clicked on page.
    2. Took maybe 10 seconds to take in what is "above the fold".
    3. Scrolled down to see what else there is.
    4. BAM! Popup triggered by scrolling down.

I will say, generally, when I'm to the point that I'm entering credit card info, I've put up with it, but I have been chased off of sites by this use case before. Especially if that popup also crosses with some other popup and now I'm chasing down the tiny little 6pt light-grey-on-white little "x"s to click away the popups in the right order.

Actually, let me add that to my touchstone list. OF COURSE hiding the dismissal icon for the popup increases "engagement" with the popup. You don't even need to run a test for that, because what other result could it have? "We shrank the close icon, moved it to the lower right corner where nobody expects it, and made sure to kill the constrast even harder, and customers dismissed it 2.5 seconds more quickly on average"? Of course that's not possible. But... that's the wrong question! And AB testing is really good at answering the question you're asking, it has no mechanism in and of itself to see whether you're asking the right question. If you're getting down to this you've overoptimized.

I was just pontif- er, talking about this to someone, a couple of days ago.

I love the users of my products. Most of my products are free, and are carefully-crafted, highly-polished, complete deliverables, and I fret over how they are used -even if by a tiny number of end users-, like a nervous hen. I do what I do, out of love for the craft, and out of a genuine desire to make people's lives easier, through the technology I have at my disposal.

It is my belief that most tech companies despise their user base. Users are little more than cattle, to be fattened and slaughtered. "Caring about the user" means optimizing for "engagement," or keeping them trapped within their own ecosystem. John Oliver did a rant about this, recently[0]. It has nothing to do with actually caring about the user, or solving their problem. It is about harvesting users.

In fact, my discussion about this, came about, because someone wanted to keep users inside the app I'm writing, as opposed to linking them to a more familiar app, on their phone (for the record, it was for videoconferencing). Linking is a "no-brainer," as I can link out to dozens of installed apps, using the simple URL scheme method, built into iOS[1], and "keeping them in the app," would have required several months of extra work, polluting the app with megabytes of junk code, because I'd need to use SDKs, and also kill the ability to easily scale to add new clients (contrary to popular belief, Zoom is not the only videoconferencing option). It would also have possibly put us on the hook, legally, for what happened in those videoconferences.

[0] https://youtu.be/jXf04bhcjbg?t=638

[1] https://developer.apple.com/documentation/xcode/defining-a-c...

If it's something one-and-done (like different permutations of a signup flow to see what is easier for users), then I don't see the harm in it.