271–280 of 346 posts
Earlier quoted context omitted.
I think of it as validating and improving imperfect user assumptions.
This reads exactly like parody corporatespeak jargon.
Earlier quoted context omitted.
My 'favorite' silly thing PMs do is UX research studies (typically on 5-10 people) and essentially ask completely untrained people if we should go with X/Y or Z. It's a super-effective way of avoiding responsibility for product decisions ("the data suggest we should go with Y"). If only building good products were as easy as asking what customers think they want.
Either they're doing the UX research wrong or (more likely) you're misunderstanding the process. You don't ask them if you should do X/Y/Z. You ask them to do X in the program, and see that none of them can find widget Y which controls it because they keep clicking on widget Z. It's about observing the users fumble through your UX when you know their motivation.
Some time ago we did such a test. We called 10 customers to our offices and had them do some flows in the application. They didn't fumble. They pretty much did what they had to do and left positive reviews.
That whole thing got scrapped because consultants convinced our CEO that qualitative data is not good for global scoped startups, and that we should be building based on quantitative data.
Honestly, in less than a year, our customer experience was already taking a dive because all the extra little features we would add and strange UI elements, it became a confusing mess and our tracked NPS (Net Promoter Score) showed that. I've since left the company, but I check on them from time to time and they never really recovered and continue doing A | B in the hopes of hitting that sweet spot. It's just an unrecognizable monster at this point in my opinion.
Earlier quoted context omitted.
Email marketing in general blows my mind. Marketers typically have absolutely no respect for consent, and the costs are completely borne by the recipient. The whole industry depends on dark patterns, shady list sharing, and scraping your email to add it to their lists despite you having no relationship with them. I know it's not simple, and it's just my frustration speaking, but I don't understand how my mail host ca…
beware that Meta/FB and Tiktok scripts are among those that siphon off email data even before a web form is submitted https://arstechnica.com/information-technology/2022/05/some-...
Earlier quoted context omitted.
You are correct. I will figure out two things. 1. Why our fancy expensive mailing/marketing tool (Intercom) does not spread these messages in a relaxed fashion (it should) 2. If of those five messages, maybe two were the obligatory "confirm your email" and standard "Welcome to Checkly, this is what we do" TL;DR we should not spam.
We checked our Intercom. We spammed. One user got six emails in a 5 day span. There were some separate initiatives going on. We didn't check the settings and current outgoing mail. We will change it. BTW: all of this was done without any bad intent. It's 100% us being stupid and not coordinating and being diligent enough.
Anecdotal: we released plenty of improved features, like a better gallery to see the items in our shop, users used it a lot +250%, but conversion rate went down 4%. They spent more time seeing the items and..didn't like the pics and conversion went down. In the end we reverted to the crap gallery we had before, they don't click it anymore and conversion went back up again..
If it possible that there'd be a long term effect like: * Users know you have a nice gallery * They are more likely to shop at your store * In the end, you get more sales despite the lower conversation rate
Earlier quoted context omitted.
In fact, you want the dismiss button to be easily discovered and used. Dismissals are an important signal about the quality of the content; just as important as clicks. When you make the button impossible to use you rob yourself of that signal while simultaneously making click data far less reliable.
Unless you're a manager, not getting the results desired. Then the problem is people, and "they're just clicking close out of reflex!". Cue hiding this, and results you wanted appear! Success! Raise! Promotion! Or, maybe more funding, due to signs of greater engagement.
Earlier quoted context omitted.
Better customer experiences often times lead to increased metrics. They are not totally different things.
Maybe if you are very aware of the fact that your goal and metric aren't totally aligned, but that often gets lost. As a result A/B testing for longer website visits can make websites that make it more obvious that the information you want is there, but also make the path to actually get it longer. A/B testing for engagement might promote divisive behavior and fights. A/B testing for read rate or clicks might lead to…
I really don't like the positioning of ALL A/B testing as unethical behavior where you're hostilely trying to take advantage of a user. It's quite the opposite. There are a lot of extremely poor user experiences out there and a quality testing program can help improve user experiences, remove risk from making sweeping changes, and help you learn more about your audience and market.
The vast majority of the successful testing I've done is done around trying to HELP users navigate the site and product catalog, understand the product, and purchase the product. Attention spans are fleeting with online shopping and even the smallest points of confusion or friction can turn shoppers off.
Additionally, often times I'll read into test results after a month or so to see if there were any issues with orders that might indicate purchases from disinterested people or misaligned expectations.
Earlier quoted context omitted.
Have to disagree. I've found plenty of issues that affect real production users through the use of AB testing. Problems that were small enough to escape review, testing, and reporting, but large enough to be stat-sig. They always lead to a bug, or issue with test vs control. I will always use AB testing for uncertain code in the future. I was skeptical when I first started writing AB tests, but they have proven their…
Sure, but that's not really A/B testing, those are more often called staged rollouts or progressive rollouts.
These types of experiments are extremely valuable in uncovering hard-to-find bugs, assuming you have sufficient logging and confidence around your metrics. They let you know a problem exists and roughly where it is in the product. From there you can drill down and investigate your source code until the discrepancy is found.
Earlier quoted context omitted.
You are correct. I will figure out two things. 1. Why our fancy expensive mailing/marketing tool (Intercom) does not spread these messages in a relaxed fashion (it should) 2. If of those five messages, maybe two were the obligatory "confirm your email" and standard "Welcome to Checkly, this is what we do" TL;DR we should not spam.
We checked our Intercom. We spammed. One user got six emails in a 5 day span. There were some separate initiatives going on. We didn't check the settings and current outgoing mail. We will change it. BTW: all of this was done without any bad intent. It's 100% us being stupid and not coordinating and being diligent enough.
"As an experiment, I went through a list of holiday weekend sales, and opened all the sites. They all — all, 100% — interrupted my attempt to give them some money." This is a good touchstone to use for "you've overoptimized your site, tone it back". I am also taken aback every time I'm on a site, I've got something in my shopping cart, I'm headed for the "check out" button, or I'm even on the checkout page , and some…