Earlier quoted context omitted.
Speaking of 'sign up to our newsletter', one of the latest dark patterns I've found that astounded me was adding a checkbox to the login form [0], where you'd normally expect the 'remember me' checkbox to be. You almost click it out of muscle memory if you don't read what it says. [0] https://i.postimg.cc/HW89hs7r/Screenshot-2022-07-12-145957.p...
atlassian does this as well: https://i.postimg.cc/zfwbG5Ft/atlassian-login.png
A/B testing gets misused to juice metrics in the short term
291–300 of 346 posts
Re: A/B testing gets misused to juice metrics in the short term
#292Earlier quoted context omitted.
Mostly, it doesn’t even seem to matter whether you agree or not. Inevitably you end up receiving affiliate emails regardless.
This is why every company gets a different address for me. If junk starts coming in, that address is blocked, and I stop doing business with that company (if I haven't already). This sometimes falls foul of spammers adding some random addresses of the form blahblah@mycatchall.domain.tld or @mycatchall.domain.tld into their lists, but that hasn't happened often enough to be a problem. That it isn't much of a problem s…
@mydomain.tld
Spelling the company's name backwards makes it easy to match to a company for use by my own spam filter without setting off their pattern detectors.
Re: A/B testing gets misused to juice metrics in the short term
#293Earlier quoted context omitted.
> MBA's are generally better communicators and much easier to get along with. This is exactly the problem. The MBA skill set is climbing corporate hierarchies. Not product design, customer service, empathy, or long-term thinking; only promotions. That's the whole reason anyone gets an MBA. MBAs often lack domain knowledge, but even when they have it, the MBA problem remains because their priorities are unrelated to p…
"You can tell the witches by their propensity to gather with one another - which is proof indeed they are witches, and must be burned, lest they infect the rest of us good people!" My. God. Man. The amount of misunderstanding, arbitrarily 'made up stuff' and naive speculation on this thread ... it's just ridiculous. It's kind of general professional designation, and valuable education for those with no business backg…
> the language of finance, macro/micro economics, marketing
This is like saying people get computer science degrees to learn about programming. Is it important for the job? Yes. Will you learn it in the program? Yes. Is it why people pursue the degree? Absolutely not. An MBA is an expensive, time-consuming certification. Nobody gets a graduate degree in business administration because they enjoy the classes. People pursue them because they provide opportunities for career advancement at the higher levels.
> Some of them are good, some are not.
This is the core of the problem. For a company, it doesn't matter whether the MBAs are good, because they're almost definitionally climbers and as climbers their incentives are fundamentally misaligned with long term success.
I'll use Boeing as an example again, because it's such a textbook case. Management (composed primarily of MBAs) discovered senior engineers were expensive and laid them all off. Twenty years later, their planes crash, their capsules leak, and their rockets don't work well enough to launch at all. Did they make the wrong decision? Well it looked very good on a lot of quarterly revenue reports, EBITDA was up, and the CEO responsible for most of those layoffs (James McNerney, MBA) retired well before the disasters to accolades, trusteeships, and not tens but hundreds of millions of dollars. From his perspective, he achieved the goal behind getting an MBA in the first place.
There's nothing wrong with ambition, but being wary of people with too much of it has been a tenet of humanity since before the refinement of bronze.
Re: A/B testing gets misused to juice metrics in the short term
#294You can get long term results from AB tests long after the test has ended... For example, you can see if Group A or Group B from a test are more likely to still use the site 1 year later. You hypothesize that those ways to 'juice the metrics in the short term' hurt the user experience in the long term... Well if your hypothesis is right, these long term AB results should show it.
This isn't very feasible on most products and certainly limited by the amount of data collected.
Re: A/B testing gets misused to juice metrics in the short term
#295Re: A/B testing gets misused to juice metrics in the short term
#296If you're upset about internet retail, I hope you're also upset about milk being in the back of the store to get you to walk through the whole thing, because this has been merchandising's bread and butter for a very long time.
Re: A/B testing gets misused to juice metrics in the short term
#297Earlier quoted context omitted.
Sure, but that's not really A/B testing, those are more often called staged rollouts or progressive rollouts.
I'm talking about running week or month long tests with control and multi test cells containing new functionality, configuration, or code to determine the viability of a single or combination of changes by analyzing statistical output driven by p-value and pre-determined target metrics. These types of experiments are extremely valuable in uncovering hard-to-find bugs, assuming you have sufficient logging and confiden…
Re: A/B testing gets misused to juice metrics in the short term
#298Earlier quoted context omitted.
Recently I got an account on a developer tool (Checkly) because the company I joined uses it. I then got 5 different emails from them in a 48 hour period. Like I'm sure many users sign up then drop out of your funnel but I'm part of an organization that's a paying customer. I'm already going to use your stuff. What possible business benefit could there be to you spamming me? If anything you're risking the inverse - i…
Hey CSMastermind, I'm founder at Checkly and I got a ping we were mentioned. We do send out some "getting started" messages on autopilot. We also did a product launch Thursday and then our regular changelog on Monday. That probably was overwhelming. If you could email me on tim -at- checklyhq -dot- com I will track down if we hit the the spam cannon too hard.
Maybe consider, oh I don't know, not deploying a spam cannon in the first place?
The good news is that this sort of thing is enough to trigger the spam filter in my email program so I'll never see it.
Companies have this astonishing arrogance assuming they should be able to interrupt actual humans with robot messages from noreply@crapcorp.com - typically marketing messages that cannot actually be replied to and do not qualify as respectful communication.
Re: A/B testing gets misused to juice metrics in the short term
#299Earlier quoted context omitted.
This comment is way too broad and cynical. It reminds me of the sentiment I sometimes hear from my teenage children before I explain to them that the world isn't a Bernie Sander/Reddit gotcha soundbite and that reality is complicated.
Let me be even broader and more cynical: everything a company does is ultimately to maximize profit.
Re: A/B testing gets misused to juice metrics in the short term
#300I've talked about this on HN but I'll say it again. A couple years ago, after being an Android fan for the better part of a decade, I finally bought myself an iPhone and pried myself away from Google's ecosystem wherever I could. And Apple didn't even need to do any work for me to make this decision. It was the years of abuse from Google that you experience when you decide to use a Google product or service. And a bi…
"To me, A/B testing means you don't respect your users. You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. " Your perspective is extremely short-sighted. A/B testing can result in this type of behaviour but that's just poor A/B testing. Good A/B testing focuses on removing distractions from the experience and helping users derive…