Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A/B test of banner ads vs. traffic

gwern.net

Post reply on HN
Print view View original

1–10 of 149 posts

Re: A/B test of banner ads vs. traffic

#3

Post by katzgrau »

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking.

Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is.

That may sound obvious, and it probably is. But the general effect is that users engage with and are sometimes genuinely interested in banner ads for their favorite restaurants, venues, and other spots. Local and niche advertisers are part of the community. A Ford or Chevy ad is generally much less relatable.

So my overall point is that the nature of the publication and its advertising clients is going to impact whether some readers are turned off - so it's difficult to derive a rule of thumb for banner advertising as a whole.

Re: A/B test of banner ads vs. traffic

#5

Post by Terr_ »

katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

I imagine the worst advertising is where it feels both unfeelingly-impersonal yet also spyingly-specific.

Re: A/B test of banner ads vs. traffic

#6

Post by gwern »

yellow_postit wrote: post #4

Unless I misread this it sounds like the assignment is on 2 day chunks rather than some unique user ID. Just doing this at impression rather than user level makes me question the applicability of the results, but really interesting read nonetheless.

Per-user level doesn't capture the effects I am interested in. Imagine you click on a website and the banner ad makes you subconciously less happy and so you don't submit it to HN for thousands of other users to read; how would randomizing on you and thousands of others' user IDs help? There's no way to link your user ID's randomization to the subsequent visits (or non-visits) of thousands of other people; it just averages everything out to 0 ('500 visitors were exposed to ads, 500 were not; conclusion: ???'), when in fact the real answer is something like '-1000 hits because user yellow_postit was exposed to ads and didn't like them'.

But if you did it on a per-day or per-week or per-month basis, then you could see effects like that: "on ad-free weeks, there are twice as many submissions to Reddit which drives +100 hits per day" etc. Then you don't need to try to track each individual user's activities or reshares - whatever the effects are, they'll show up at the population level.

Re: A/B test of banner ads vs. traffic

#7

Post by stemuk »

katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

Maybe a site like gwern.net is generally less suited for A/B experiments like that because of the unique look and speed 'HTML only' sites have. Sites like cnn.com or yahoo.com are generally much more crowded and colorful, so banner ads blend in much better. This might in return result in a higher user acceptance rate for banner ads and probably a different engagement rate too.

Re: A/B test of banner ads vs. traffic

#10

Post by HugThem »

katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

Where do your ads appear? On your site I do not see a publisher signup page.
Post reply on HN