Live data from Hacker News

A/B test of banner ads vs. traffic

gwern.net

41–50 of 149 posts

Re: A/B test of banner ads vs. traffic

#41
post #16

I honestly couldn't follow most of the math but found this really interesting. I used to do UX analyses at a marketing agency and would constantly link clients to [1]. Page speed is huge, especially on desktop where its easier to navigate away. OP mentions this, but I imagine the effect is massive on a site where users are actively researching a topic and have tens of tabs open and so can easily switch to a similar w…

In my experience that kissmetrics blog is a load of nonsense in practice. We massively reduced load time on the all the key pages from 4 secs to roughly I did a similar thing with their mobile site a year or two later, massively reducing the page-weight and reducing loading times on the devices I tested from seconds to again sub-second load times. Again no discernible bump.

Did you try to see what happened if you increase the load time above 4 seconds?

Re: A/B test of banner ads vs. traffic

#42
post #38
post #33

Earlier quoted context omitted.

The other option is to target the ads to the publication . As long as the publication is on a niche topic or geographically localized, you can choose ads that will likely interest people (local restaurants) without needing to track them. Tracking is only necessary when the ad venue is incredibly broad across topics and geography.

I have been wondering why that isn't done more often myself. I mean,it isn't like tv ads didn't work.

Advertising is very much beset by the mindset of being "so preoccupied with whether or not they could that they didn't stop to think if they should" (Jurassic Park.)

Tracking and microtargeting just seem too attractive for advertisers to pass up. Same goes for large size, autoplay, multimedia ads.

These mechanisms would have to fail to perform in obvious ways (not long-term, subtle, whole-market ways) before people will stop using them.

Re: A/B test of banner ads vs. traffic

#44
What I learned from running hundreds of forums is that:

1. Banners hurt unless they are relevant

2. Filtering for banners that are relevant tends to reduce revenue rather than increase as there is a smaller pool of highly relevant advertisers

3. Removing all banner adverts significantly increased engagement by every metric

4. Some revenue is possible using affiliate links within the content (re-writing existing links rather than inserting awful links on a word match basis)

That last one is what I now do. Is it as profitable as running banners everywhere? No. But it is the highest possible engagement with some revenue, and if one focuses energy on reducing costs instead it can work well.

Re: A/B test of banner ads vs. traffic

#45

There are ways to have comments that are a lot less awful than Disqus.

I scanned the article to see if it was interesting enough to dive into the details, but when I saw the Disqus plugin I closed the tab. What's the point of dropping Adsense if you keep the Disqus plugin?

Re: A/B test of banner ads vs. traffic

#46
post #6

Unless I misread this it sounds like the assignment is on 2 day chunks rather than some unique user ID. Just doing this at impression rather than user level makes me question the applicability of the results, but really interesting read nonetheless.

Per-user level doesn't capture the effects I am interested in. Imagine you click on a website and the banner ad makes you subconciously less happy and so you don't submit it to HN for thousands of other users to read; how would randomizing on you and thousands of others' user IDs help? There's no way to link your user ID's randomization to the subsequent visits (or non-visits) of thousands of other people; it just av…

Isn't this particular test susceptible to confounding effects? Traffic fluctuates day-to-day, week-to-week, and month-to-month, so how can you be sure it was the presence-or-absence of ads and not something else? If you randomize at the visitor level, you are sampling from both high-and-low traffic days, and control for any external fluctuations.

Re: A/B test of banner ads vs. traffic

#47
post #43

How does this experiment methodology account for cross device mixed experience? Because of the multiple devices use, There will be lot of users that have seen both variation.

I think changes over time in device should be like any other unobserved variable like age or country or gender which might affect traffic: averaged out by randomization. The point of randomization is that on average, the two groups will be balanced. (If the intervention itself changes the mix and the changed mix then yields more traffic, then that changes what one might speculate about how it works but the simple result of ads->less-traffic is still true.)

Re: A/B test of banner ads vs. traffic

#48
post #39
post #19

Earlier quoted context omitted.

Good question. Even if it does not load, it should still show the adsense code in the html. But looking at some random post, I don't see it: http://web.archive.org/web/20150304235943/http://www.gwern.n... Adsense is usually a script loaded from googlesyndication.com which is not in the source here. Maybe archive.org rewrote the html so much that it is not there anymore?

After a few moments of searching I'm going to assume that whoever specifically scraped and uploaded the page may have removed the adsense javascript. Doesn't look to be a normal archive.org thing.

You can upload pages to archive.org? Wouldn't that open the door for spam and other abuse?

I checked some other site and the Adsense code is not changed by archive.org. So something is strange.

Maybe Gwern experimented with different types of ads and on that page at that time did not use adsense?

I do see the adsense code in a snapshot from a year later:

http://web.archive.org/web/20160629230243/https://www.gwern....

Re: A/B test of banner ads vs. traffic

#49
I would love to remove ads from all but one of my sites. But I have not been very effective at monetizing my writing some other way. I removed them from a bunch of sites and saw a decrease in ad revenue, but did not see some noticeable difference in either traffic or tips. (I only later added Patreon.)

There are myriad contributing factors that I can't isolate out. People don't like ads, but many of the people who loudly complain about ads will also very aggressively tell you that they will not tip, will not be a Patreon supporter and expect high quality content for free. They will aggressively say that your business model is your problem, not theirs, while actively shooting down any way to monetize content.

These are often people who make good money as, for example, programmers. Another thing such people have told me is to get a real job, like they have, and write for free. It is a kind of classism in a sense: Writers who provide good quality web content aren't seen as valuable or serious contributors the way programmers are. Yet, at the same time, people decry both the lack of good content and the trend towards content done as content marketing.

I get paid to write content done as content marketing. I rarely write anything I feel bad about. I work on projects I can believe in. But I find it frustrating because I would rather spend more of my time producing high quality, independent content for my own projects. These are projects intended to put out useful information and/or to be entertaining.

I am doing some unique things that add value to the world. I run a homeless site with useful information for homeless Americans. Most homeless services have terrible websites. They are almost always donor facing, not client facing. It is very challenging for a homeless person to find useful information online. I had a college class on internet search and I had a hard time finding information I needed.

The other thing is that a lot of homeless services are a very negative experience and can help keep you trapped. Services are often designed with the idea that homeless people are incompetent losers who can't be trusted to make good decisions. They often come with strings attached that can help keep a person trapped in poverty.

I want to promote market based solutions that can work for homeless individuals, but can lead to a future because they continue to be viable after you get off the street. Programs to help the homeless or help poor people often actively encourage failure.

I think I have worked out a couple of models that work, but I am not finding a path forward. I continually come up against the idea that I should just stop caring about marginalized people because there is no money in it. I should just go where the money is, and never mind that the world is going to hell.

And I can't quite bring myself to do that because there is scary stuff happening and I feel like if I don't do something, no one will. Because rich people aren't going to solve it. The attitudes of so many of them that poor people just need a handout (UBI), not earning opportunities, is part of the problem.

We have the technology to allow people with disabilities, like me and my sons, to work in a way that works for us. But I don't know how to overcome the attitudes that we shouldn't bother to try, we should just accept that we are losers unworthy of any respect.

The inability to get taken seriously is a big barrier to getting engagement and traction and that is a big barrier to turning it into money. The whole thing makes me crazy. It is a case of the key is in the safe.

Re: A/B test of banner ads vs. traffic

#50
post #33
post #26

Earlier quoted context omitted.

As a user,I am most offended by ads that target me. I would rather see a completely unrelatable ad. It feels like learning new information that way,as opposed to being stalked and preyed upon.

The other option is to target the ads to the publication . As long as the publication is on a niche topic or geographically localized, you can choose ads that will likely interest people (local restaurants) without needing to track them. Tracking is only necessary when the ad venue is incredibly broad across topics and geography.

That's how old media has been doing it for centuries. But media is hard: you have to invest in personal relations with advertisers.

New media just outsources all the hard work to Google and other big ad placers. Easier but you lose editorial control over your work.

Post reply on HN