Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A/B test of banner ads vs. traffic

gwern.net

Post reply on HN
Print view View original

51–60 of 149 posts

Re: A/B test of banner ads vs. traffic

#51

Post by badestrand »

buro9 wrote: post #44

What I learned from running hundreds of forums is that: 1. Banners hurt unless they are relevant 2. Filtering for banners that are relevant tends to reduce revenue rather than increase as there is a smaller pool of highly relevant advertisers 3. Removing all banner adverts significantly increased engagement by every metric 4. Some revenue is possible using affiliate links within the content (re-writing existing links…

Thanks for sharing. I really wish there was a general web subscription with revenue-sharing between all visited websites. I hate seeing the ads on my own sites but have no viable way to replace them. And as a user I would like to support several sites, apart from disabling the adblocker.

Re: A/B test of banner ads vs. traffic

#53

Post by ar0 »

Feniks wrote: post #50
tikhonj wrote: post #33

Earlier quoted context omitted.

The other option is to target the ads to the publication . As long as the publication is on a niche topic or geographically localized, you can choose ads that will likely interest people (local restaurants) without needing to track them. Tracking is only necessary when the ad venue is incredibly broad across topics and geography.

That's how old media has been doing it for centuries. But media is hard: you have to invest in personal relations with advertisers. New media just outsources all the hard work to Google and other big ad placers. Easier but you lose editorial control over your work.

In fact, that’s how Google started out with AdSense as well: It was called “content targeting advertising”, because Google scanned your webpage and then served ads that related to the content of your webpage.

I don’t know what happened along the road to lead us to the current tracking and spying mess.

Re: A/B test of banner ads vs. traffic

#54

Post by smsm42 »

katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

> It turns out that the more relatable a banner ad is, the less offended the reader is.

This is of course true, and kinda obvious, but I am not sure this is workable as business model. I mean, here are not a lot of ads that are immediately relevant for me. Maybe once or twice per month there's something that I buy that ad could sway my decision at least potentially (i.e. expose me to some provider I wasn't aware of, or some brand I haven't considered). The rest is basically wasted on me. How much 2 clicks per month (with unknown conversion rate) are worth to advertisers? I have no idea, but I wonder whether it is possible to support a site on that revenue?

I, of course, could be wildly atypical case (for one, I use adblockers and rarely impulse-buy) and maybe typical visitor would be much more engaged with ads.

Or it could be that ad buyer industry (not ones that make the ads, but ones that order them - Fords and Chevys of the world) do not have proper tools to see how effective their ads are and just throw a chunk of budget to the wind, get some traffic from it and feel good about it.

Re: A/B test of banner ads vs. traffic

#55

Post by icelancer »

mattmanser wrote: post #24
soared wrote: post #16

I honestly couldn't follow most of the math but found this really interesting. I used to do UX analyses at a marketing agency and would constantly link clients to [1]. Page speed is huge, especially on desktop where its easier to navigate away. OP mentions this, but I imagine the effect is massive on a site where users are actively researching a topic and have tens of tabs open and so can easily switch to a similar w…

In my experience that kissmetrics blog is a load of nonsense in practice. We massively reduced load time on the all the key pages from 4 secs to roughly I did a similar thing with their mobile site a year or two later, massively reducing the page-weight and reducing loading times on the devices I tested from seconds to again sub-second load times. Again no discernible bump.

Right - the pageload time only matters for certain sectors. We sell sports science equipment and our competitors have 10s load times; I'm sure they have no issue selling their stuff. If they got their load times down to ~1s I bet they'd sell more... but not that much more. We are in the 1.5s-2.0s load time and I'm happy with that; given we use Woocommerce to drive our traffic I highly, highly doubt I can get it much lower given our current site design which does very well engagement-wise. That's fine for us; our conversion metrics and related analytics show that we're doing very well for our industry.

For niche equipment you're either gonna buy it or you aren't. If it isn't Amazon, Google, Wal-Mart, or places where you are preying on impulse buys (Woot, LowEndBox/VPS, etc), then the page load times matter a hell of a lot less than something like user design.

Re: A/B test of banner ads vs. traffic

#56

Post by gwern »

portman wrote: post #46
gwern wrote: post #6

Earlier quoted context omitted.

Per-user level doesn't capture the effects I am interested in. Imagine you click on a website and the banner ad makes you subconciously less happy and so you don't submit it to HN for thousands of other users to read; how would randomizing on you and thousands of others' user IDs help? There's no way to link your user ID's randomization to the subsequent visits (or non-visits) of thousands of other people; it just av…

Isn't this particular test susceptible to confounding effects? Traffic fluctuates day-to-day, week-to-week, and month-to-month, so how can you be sure it was the presence-or-absence of ads and not something else? If you randomize at the visitor level, you are sampling from both high-and-low traffic days, and control for any external fluctuations.

Because you're randomizing at the 2-day level, on average there will be just as many advertising/high-traffic days as advertising/low-traffic days, and as many no-advertising/high-traffic days as no-advertising/low-traffic days. The randomization is unaffected by traffic and uncorrelated with it. The unit of analysis is each day, not each visitor. This is why it has to be run for several months, otherwise you don't wind up with a decent n=50 pairs.

That's the tradeoff here: it lets you look at the totals, but it takes a lot longer than if you randomize per visitor in which case you could finish the test in a few days, often.

Re: A/B test of banner ads vs. traffic

#58

Post by smsm42 »

HugThem wrote: post #8

He seems to not show how the ads looked like on his page. He seems to only attribute it to load time. But maybe the ads were visually annoying? Why no screenshots?

I am yet to see any ad setup that guarantees they are not visually annoying. Most (reputable) ad networks guarantee they filter ads and remove the evil ones. I haven't seen one that delivered. I do not see how they could - manual review of each ad is not scalable and advertisers would not abide by it, they'd just move to network with less stringent rules.

Re: A/B test of banner ads vs. traffic

#59

Post by pbhjpbhj »

badestrand wrote: post #51
buro9 wrote: post #44

What I learned from running hundreds of forums is that: 1. Banners hurt unless they are relevant 2. Filtering for banners that are relevant tends to reduce revenue rather than increase as there is a smaller pool of highly relevant advertisers 3. Removing all banner adverts significantly increased engagement by every metric 4. Some revenue is possible using affiliate links within the content (re-writing existing links…

Thanks for sharing. I really wish there was a general web subscription with revenue-sharing between all visited websites. I hate seeing the ads on my own sites but have no viable way to replace them. And as a user I would like to support several sites, apart from disabling the adblocker.

Have you tried Flattr, I've not used it but I see it on a lot of sites and gather it fits that niche.

https://thenextweb.com/tech/2017/10/24/flattr-now-lets-pay-c... may be of interest too.

Re: A/B test of banner ads vs. traffic

#60

Post by austinhutch »

mattmanser wrote: post #24
soared wrote: post #16

I honestly couldn't follow most of the math but found this really interesting. I used to do UX analyses at a marketing agency and would constantly link clients to [1]. Page speed is huge, especially on desktop where its easier to navigate away. OP mentions this, but I imagine the effect is massive on a site where users are actively researching a topic and have tens of tabs open and so can easily switch to a similar w…

In my experience that kissmetrics blog is a load of nonsense in practice. We massively reduced load time on the all the key pages from 4 secs to roughly I did a similar thing with their mobile site a year or two later, massively reducing the page-weight and reducing loading times on the devices I tested from seconds to again sub-second load times. Again no discernible bump.

These kinds of stories don't get made into blog posts because they don't help sell any kind of narrative. Novice eCommerce store owners will focus on the "top 10 things you can do to increase conversion" from blogs like the one mentioned above.

At a large scale page load time would have a discernible impact on your bottom line, but for most it is a premature optimization (as long as you aren't trying to deliver 10mb webpages).

Post reply on HN