Uber discovered they’d been defrauded out of 2/3 of their ad spend • Hacker News Board

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#71

Post by rm999 » Sun, Jan 03, 2021, 7:42 PM UTC

Back in the early 2010s I worked in ad tech on a data science team, and one of the things we were pushing for was causal A/B testing; basically turn off a campaign's advertising to a % of people and correlate it with sales to measure ROI.

As we were kicking this off I was at a conference chatting with an executive at another ad tech company. His response: "oh yeah I know a guy who tried that, he's not in the industry anymore."

We almost immediately came to realize our launch clients were getting negative ROI, sometimes severely so. AFAIK our efforts fizzled out, and I believe none of the people on my team are in the industry anymore.

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#72

Post by lumost » Sun, Jan 03, 2021, 7:42 PM UTC

jedberg wrote: post #20

There was a freakonomics podcast recently about advertising (online and traditional). No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior. https://freakonomics.com/podcast/advertising-part-1/ https://freakonomics.com/podcast/advertising-…

It's unclear how this experiment would be done. In the case of brand advertising, it's likely that brand awareness would decay over some period of time and in turn purchase behavior would change.

It's not currently possible to run an A/B experiment with a hold out group of potential customers across all channels, let alone for any longer duration experiment. So how can we separate cause and effect? (although pay per conversion channels do get gamed left and right)

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#73

Post by st1x7 » Sun, Jan 03, 2021, 7:42 PM UTC

onlyrealcuzzo wrote: post #50

You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another. You don't have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty. All big companies do this. Sure, you're never going to know exactly how many people you advertised to would have organically, eventually…

> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn't seem that important compared to the other metrics...

Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#74

Post by josefx » Sun, Jan 03, 2021, 7:42 PM UTC

birdyrooster wrote: post #23

Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it. That’s different than showing a person an ad and changing their behavior, but to advertisers they can’t know unless they experiment.

> Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it.

Seems to be entirely subjective.

I bought a new blade for a bread cutting machine on amazon, the last one lasted ~25 years. My current recommendations: blades for bread cutting machines.

I also checked some Christmas decorations on Amazon, but didn't buy any. So next to the blades I get Christmas decorations recommended. January is a bit late for that, isn't it?

I also bought a single server rack on eBay. Now I regularly get spam emails from eBay reminding me of new offers for server racks. Even the company I work for doesn't have that many, so why?

In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#75

Post by sosborn » Sun, Jan 03, 2021, 7:42 PM UTC

arbuge wrote: post #37

aetherson wrote: post #11

Earlier quoted context omitted.

In fairness, I've worked at several places that engaged in this practice, and they're perfectly clear that they're paying for clicks that they'd likely get for free. The theory is that if they don't do this, their competitors buy this ad space and steal some of their organic traffic, and/or that they're just bidding up the cost of that space to make their competitors less efficient. Now, that theory may not be cost-e…

That theory can easily be tested by turning off the ads in question and seeing what actually happens. Can always turn them on again later if needed.

Would you be willing to make that decision knowing full well that it could reduce revenue that is 100% traceable to that decision? I mean, obviously there are ways to make this politically acceptable in your organization, but you better be completely confident that it won’t bite you in the ass.

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#76

Post by notional » Sun, Jan 03, 2021, 7:42 PM UTC

jedberg wrote: post #20

There was a freakonomics podcast recently about advertising (online and traditional). No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior. https://freakonomics.com/podcast/advertising-part-1/ https://freakonomics.com/podcast/advertising-…

Why does reddit have ads if all ads are a money pit?

I understand you no longer work there, but ads started in 2009 I believe, so you perhaps had some input on this?

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#77

Post by cbsmith » Sun, Jan 03, 2021, 7:43 PM UTC

cortesoft wrote: post #60

kmeisthax wrote: post #43

I think a lot of this was already obvious to slightly-savvy mobile app users. Look at pretty much any free-with-ads app; a majority of the ads are going to be ads for other free-with-ads apps, many of which you already have installed. The thing is, outside of the Internet most ad spend is for marketing ubiquity, not direct response. You don't really buy, say, TV advertising with the expectation of getting so many cli…

The point of traditional advertising IS pretty much to waste money, similar to how the point of a peacock's tail feathers are to waste resources... it is a signal to potential partners (business or romantic) that they are so capable they can waste resources. It is a signal of bonafides.

It's also an important signal to consumers.

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#78

Post by chrisdinn » Sun, Jan 03, 2021, 7:44 PM UTC

Hnrobert42 wrote: post #54

kbar13 wrote: post #36

Earlier quoted context omitted.

if you read the twitter thread we see that uber was spending 10x more money than they should have on ads simply because they weren't auditing where their money was going towards. so there's evidence that people (incl uber, a market leader) do indeed spend money on ads even when it does nothing

I got 0.66x, as in $100M out of $150M. I likely missed something. Where did you see 10x?

$100M was labelled "fraud" in this example, but there was more waste uncovered in the remaining $50M that didn't get that label.

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#79

Post by JeanMeche » Sun, Jan 03, 2021, 7:44 PM UTC

nicbou wrote: post #56

Nextgrid wrote: post #30

Earlier quoted context omitted.

> No one is willing to run the experiments necessary The people that would have the power to run this experiment have their entire careers depending on things staying as-is. Running the experiment carries a significant risk of exposing that the advertising operations they're responsible for provide much less ROI than they pretend it does. The unwillingness of anyone to run such an experiment is already an answer. Why…

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!"

~ Upton Sinclair

Re: Uber discovered they’d been defrauded out of 2/3 of their ad spend

#80

Post by ABeeSea » Sun, Jan 03, 2021, 7:44 PM UTC

jedberg wrote: post #20

There was a freakonomics podcast recently about advertising (online and traditional). No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior. https://freakonomics.com/podcast/advertising-part-1/ https://freakonomics.com/podcast/advertising-…

This is incorrect. Companies do the experiments. They just don’t publish the results. Why would they? The idea that Amazon doesn’t know the ROI on their ad/marketing spend is laughable.