What's really stupid are sites from which you can buy things, but then pop up an ad for something else. Fandango, which sells movie tickets, does this. As you're trying to get to the "buy ticket" page, they shove movie trailers for other movies in your face. I mentioned a site earlier today which sold plumbing supplies.[1] They pop up a "gimme your email" box which 1) cannot be dismissed , and 2) isn't even theirs, i…
I blame this on A/B tests which ignore externalities. Sure, maybe the 3 metrics you were aiming for were improved, but your brand is fucked. I think brands like Sears/Kenmore/Craftsman were A/Bed to death. So many little 'optimizations' (read: cost cutting that looked-good-on-paper) that eventually destroyed what the brand was about in the first place.
Both opt-in and opt-out ways of doing surveys on your website introduce bias. The opt-out (read: in your face pop-up) is more likely to make people angry which makes them more likely to leave negative reviews.
The opt-in version will only be found by people having a relatively extreme opinion about your site, good or bad.
The difference being in the quantity of the data. The first will probably be answered by more people, since the people who'd have completed the survey in the second case are probably more or less a subset of the people who would complete the survey in the first case. All of this makes it no surprise that most websites that want to have any kind of feedback from their users go for the opt-out strategy. More is better if less is not in any way more :)