Live data from Hacker News

Throw Out Everything You Know About Ads

blog.ads.pof.com

61–70 of 128 posts

Re: Throw Out Everything You Know About Ads

#61
I think it's critical to not lose track of buyer motives and end goals when testing ads.

If I were clicking on the first ad, I would do so with the intention of downloading Need for Speed. The second ad, I would click to answer the question, "Who made this crappy paint ad? It's pretty unique."

If you are paying per click, the first ad is going to perform better and will give more relevant traffic. If you are paying per impression and just care about traffic, the second wins.

Re: Throw Out Everything You Know About Ads

#62
post #53

I think the comments in this thread about ctr vs actual conversions are a bit silly. Pof's target audience here is "people who like video games" and they're trying to get people to play a free online game. They don't really have anything to teach people in verticals where the consumer is making a measurable commitment. You do not need to be very persuasive to "convert" with that crowd. Almost all marketing advice mor…

Other pieces of universal marketing "truth", all of which are accurate/general enough to not be refuted: 1) Ethos (your perceived character) is the most important, with regards to pathos (emotion) and logos (logic) 2) People make judgments by comparison/anchoring. 3) People process information best from stories. 4) People are foremost interested in things that affect them. 5) Breaking patterns gets attention. 6) Peop…

[deleted]

Re: Throw Out Everything You Know About Ads

#63
post #53

I think the comments in this thread about ctr vs actual conversions are a bit silly. Pof's target audience here is "people who like video games" and they're trying to get people to play a free online game. They don't really have anything to teach people in verticals where the consumer is making a measurable commitment. You do not need to be very persuasive to "convert" with that crowd. Almost all marketing advice mor…

Other pieces of universal marketing "truth", all of which are accurate/general enough to not be refuted: 1) Ethos (your perceived character) is the most important, with regards to pathos (emotion) and logos (logic) 2) People make judgments by comparison/anchoring. 3) People process information best from stories. 4) People are foremost interested in things that affect them. 5) Breaking patterns gets attention. 6) Peop…

Wow great list!

I'm really jealous because I've been reading about the value of being able to generate specific examples:

http://lesswrong.com/lw/5kz/the_5second_level/

Re: Throw Out Everything You Know About Ads

#64
post #47
post #35

The "test everything" mantra sounds good, but in practice, you generally have only so much data you can afford (in impressions per day, or whatever), and when your CTRs are often 0.1% or lower, you need quite a lot of data to get narrow confidence intervals around your CTRs. Using the basic binomial model, if you have two test conditions, one of which actually does 20-25% better than the other, (say, 0.11% versus 0.0…

As someone who has spent $100,000 on advertising over the last few years, at least $10,000 of which was on the Plenty of Fish platform, you're very wrong. You only need about 1,000-10,000 impressions to get an idea of how a creative performs. Often less. As you get more and more used to each particular advertising platform, you also get a feel for how an ad is performing. In my business, a difference of 0.02% CTR cou…

I think part of the point abeppu is making is whether or not the two the statistics for each advertisment type are different. You see a difference of a 0.02% CTR but is that significant?

An analogy (from an interesting post I can't find) is tossing a coin for two samples sets of a 1000 times. In one sample I wear a read jumper and in another I wear a green jumper. I find that the green jumper gives me a 0.02% improvement in producing heads. Therefore I will always wear my green jumper in future when I toss a coin.

Obviously however this is just random error. A statistical analysis of CTR will tell you if the difference between the two advertisments you are observing is significant. The larger the samples size, as abeppu wrote, the greater the greater confidence in your results.

Re: Throw Out Everything You Know About Ads

#65

I suspect this is all about novelty . Users haven't seen an ad like this one a thousand times before. Some might not even think this is an ad. Some will be curious enough to actually click. The MS Paint banner ad is not inherently more effective. If every banner ad on the internet was hand-drawn in MS Paint in 5 minutes, the joke would quickly grow old, and the CTR would vanish.

I agree on the relevance of the novelty factor. It also came to my mind that this kind of childish MS Paint graphics are becoming somehow pop-culture, a meta-internet-meme if you want. Yesterday I spent half an hour on Christopher Poole's http://canv.as/ (just because I ain't brave enough to go to 4chan) and noticed a lot of these graphics; after some time lurking, I started considering them kinda cool. It's like... well, a brand.

Re: Throw Out Everything You Know About Ads

#66

0.137 CTR is seriously low regardless, 0.049% CTR is incredibly low!

Not for social. http://techcrunch.com/2011/11/14/page-post-ctr/ quotes an average click through rate for Facebook ads of 0.05% (not the headline 0.14% which is about Facebook Pages CTR), so 0.049% is close to average and 0.137% is an excellent CTR for Facebook ads.

Re: Throw Out Everything You Know About Ads

#67
I wonder if the inspiration for this test was directly taken from Max Teitelbaum's interview @ mixergy?

Search for 'paint' and you'll find the relevant passage.

  "Andrew: Give me an example. I’d love to. 
  When we started media buying, we saw stock raising. 
  We saw what all the brands were doing. We needed
  something really flashy, something really clean.
  When I made something in Paint, as a joke and sent it over 
  to the guys, we just threw it up as a test, and it 
  quadrupled the conversion rate of anything else. 
  After that, I think we made every banner in Paint."
http://mixergy.com/teitelbaum-whatrunswhere-interview/

Re: Throw Out Everything You Know About Ads

#68
post #43

This is definitely interesting. I know I'd be more inclined to click something that looks like that ridiculous paint drawing. But this bit bothered me: Results? 0.049% CTR vs. 0.137 CTR I hope I'm not the only one confused by this, but the lack of a percent symbol on the 0.137 means he went from 0.049% to 13%, an improvement of almost 300x. Is this a typo? Did he mean to say he increased his CTR to 0.137% or did he a…

This also bothered me. Just reading through his blog post makes it clear that proper English isn't very high on his list of priorities.

Re: Throw Out Everything You Know About Ads

#69
post #32

Besides the novelty factor (which others have mentioned), his 'paint' version is also much simpler, and therefor much easier to understand at a quick glance. - Large text - Not very much text - A single image - A single visual flow (top to bottom) - A concise color palette (greens and black) Compare to the 'standard' ad, where: - Text is too small - Too much text - Too many images competing for attention - Muddled vi…

yes. the point is: to create a successful add, what are we trying to optimize and are these the right things.

Re: Throw Out Everything You Know About Ads

#70
post #47
post #35

The "test everything" mantra sounds good, but in practice, you generally have only so much data you can afford (in impressions per day, or whatever), and when your CTRs are often 0.1% or lower, you need quite a lot of data to get narrow confidence intervals around your CTRs. Using the basic binomial model, if you have two test conditions, one of which actually does 20-25% better than the other, (say, 0.11% versus 0.0…

As someone who has spent $100,000 on advertising over the last few years, at least $10,000 of which was on the Plenty of Fish platform, you're very wrong. You only need about 1,000-10,000 impressions to get an idea of how a creative performs. Often less. As you get more and more used to each particular advertising platform, you also get a feel for how an ad is performing. In my business, a difference of 0.02% CTR cou…

Both sides are right here.

First of all I don't care how much you've spent on advertising, or how much experience you have. Human brains are hardwired to see patterns that don't really exist. If you haven't done the math then I guarantee that your intuition for what matters is way wrong.

However going the other way, people take standards from the science world into A/B testing that are not appropriate. If you're testing a ton of ideas, getting the right answer 3x out of 4, and not going far wrong most of the rest of the time is a pretty good result. It is certainly a lot better than concluding that it is too hard to test those ideas at all.

But if you have a specific idea you want tested, or if you want all ideas tested to a certain confidence, then you really, really need to either do the math, or to get someone to do the math for you. Because the human brain is a pattern finding machine, and you want real answers, not made up stuff.

Post reply on HN