Live data from Hacker News

Throw Out Everything You Know About Ads

blog.ads.pof.com

51–60 of 128 posts

Re: Throw Out Everything You Know About Ads

#52
post #35

The "test everything" mantra sounds good, but in practice, you generally have only so much data you can afford (in impressions per day, or whatever), and when your CTRs are often 0.1% or lower, you need quite a lot of data to get narrow confidence intervals around your CTRs. Using the basic binomial model, if you have two test conditions, one of which actually does 20-25% better than the other, (say, 0.11% versus 0.0…

In particular, it doesn't scale well at all with the number of choices (the familiar curse of dimensionality). You can reasonably test two variants, but if your design could plausibly vary along, say, 10 axes (not uncommon), you're going to have trouble collecting sufficient data to cover the whole 10-dimensional theoretical design space. So data-driven design can usually only be applied to a small part of the design space, typically testing a small number of alternatives.

Re: Throw Out Everything You Know About Ads

#53

I think the comments in this thread about ctr vs actual conversions are a bit silly. Pof's target audience here is "people who like video games" and they're trying to get people to play a free online game. They don't really have anything to teach people in verticals where the consumer is making a measurable commitment. You do not need to be very persuasive to "convert" with that crowd. Almost all marketing advice mor…

Other pieces of universal marketing "truth", all of which are accurate/general enough to not be refuted:

1) Ethos (your perceived character) is the most important, with regards to pathos (emotion) and logos (logic)

2) People make judgments by comparison/anchoring.

3) People process information best from stories.

4) People are foremost interested in things that affect them.

5) Breaking patterns gets attention.

6) People look to other people's decisions when making decisions.

7) People will believe things more easily that fit their pre-existent mindset. The converse is also true.

8) People handle one idea at a time best.

9) People want more choices, but are happier with fewer.

10) People decide first, then rationalize - If people are stuck with something, they will like it more over time.

11) Experience is memory, the last part of the experience is weighted heavily.

Re: Throw Out Everything You Know About Ads

#55
post #32

Besides the novelty factor (which others have mentioned), his 'paint' version is also much simpler, and therefor much easier to understand at a quick glance. - Large text - Not very much text - A single image - A single visual flow (top to bottom) - A concise color palette (greens and black) Compare to the 'standard' ad, where: - Text is too small - Too much text - Too many images competing for attention - Muddled vi…

You forget to mention the prominent use of the word Free.

People love that word.

Re: Throw Out Everything You Know About Ads

#56
post #29

I suspect this is all about novelty . Users haven't seen an ad like this one a thousand times before. Some might not even think this is an ad. Some will be curious enough to actually click. The MS Paint banner ad is not inherently more effective. If every banner ad on the internet was hand-drawn in MS Paint in 5 minutes, the joke would quickly grow old, and the CTR would vanish.

He also looked only at CTR and not CPA. The sort of people who click (due to the novelty) may not be the sort of people he can convert. His overall CPA may well be lower with the other ad.

Yes. It's your basic revenue vs. profit error reworked. CTR is a poor measure of effectiveness - a lower ctr can be better on ppc if you're converting better.

Re: Throw Out Everything You Know About Ads

#57
post #42

Good lesson on the value of in-market testing. That said, would love to see some theory and analysis about why the MS Paint ad outperformed the standard ad. My hunch is that the first ad -- while it obviously looks a lot more professional -- looks like every other banner ad on the internet. It reeks of ad-ness, and it may set off some psychological barrier to receptivity amongst viewers precisely because their brains…

I think you're absolutely right. I would wager that if every ad on the internet was changed to a five-minute MS Paint scrawl, click-through rates would rapidly go back to where they were originally, if not drop even further.

Reminds me of a 'saturday morning breakfast cereal' comic;

http://www.smbc-comics.com/index.php?db=comics&id=2490&#...

...and the commercials are just a flaming baby skull barking ethnic slurs...

Re: Throw Out Everything You Know About Ads

#58
post #49
post #17

Earlier quoted context omitted.

The most over-the-top-ugly banner ad I've seen was a "Would you like to go back to school on a grant?" ad targeting women. It had a cartoon woman in a superhero costume with a blinking siren pasted (seemingly) randomly on her head.

But did you click it?

No, but my mom probably would have, and it got my attention enough to where I can describe it precisely _and_ still remember what it was advertising, though the image and the product have nothing in common. Woah.

Re: Throw Out Everything You Know About Ads

#59
post #47
post #35

The "test everything" mantra sounds good, but in practice, you generally have only so much data you can afford (in impressions per day, or whatever), and when your CTRs are often 0.1% or lower, you need quite a lot of data to get narrow confidence intervals around your CTRs. Using the basic binomial model, if you have two test conditions, one of which actually does 20-25% better than the other, (say, 0.11% versus 0.0…

As someone who has spent $100,000 on advertising over the last few years, at least $10,000 of which was on the Plenty of Fish platform, you're very wrong. You only need about 1,000-10,000 impressions to get an idea of how a creative performs. Often less. As you get more and more used to each particular advertising platform, you also get a feel for how an ad is performing. In my business, a difference of 0.02% CTR cou…

Yes, test everything. But what exactly do you mean by "get a feel"?

Suppose your CTR is known to be either 0.09% or 0.11%, you've had 10,000 impressions, and you've got 11 clickthroughs. (This is of course the most likely number if your CTR is actually 0.11%.) The likelihood ratio between the two possibilities is about 0.81. So if you thought those two possibilities were equally probable before, you should now think it's about 55% likely that the CTR is 0.11% and about 45% likely that it's 0.09%.

So if by "get an idea" you mean something much stronger than 55%:45% then I fear you may be fooling yourself, no matter how much you've spent on advertising over the last few years. (Whether that means you should reconsider "test everything" depends on the costs of testing -- the actual cost of doing it, and the cost in running something other than the currently-believed-best version.)

And with 1000 impressions? Forget it. You expect to see 0.9 clickthroughs on average with a 0.09% CTR and 1.1 on average with a 0.11% CTR. You can probably get some extra information from (e.g.) when that expected single clickthrough happens, but it's not going to take you near to that 55:45 ratio. (I might believe 52:48.)

Re: Throw Out Everything You Know About Ads

#60
The second ad is more "entertaining" in my opinion. Also "Ad Blinded-ness" might have caused the first ad to just go unnoticed among all the other ads. Being untypical helps on the web. But also, the point is that you cannot expect patterns to predict success always. The first ad was based on a pattern. Now, if we study the second ad for a pattern it might not work as well. Man, I love the web for this.
Post reply on HN