Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook managers trash their own ad targeting in unsealed remarks

theintercept.com

Post reply on HN
Print view View original

121–130 of 134 posts

Re: Facebook managers trash their own ad targeting in unsealed remarks

#121

Post by mjburgess »

drzoltar wrote: post #45
mjburgess wrote: post #5

This shows the limitations of ML: facebook has incredible amounts of preference & behaviour data on its users; and can't even meet incredibly generic categories such as "high-earner, college educated, etc.". The reason we think we're in an "AI" boom is 90% these ad. companies hyping their own abilities (an identical strategy to that of the initial boom in the 50s). What we call "AI" today is just an associative house…

I’m a bit confused why we are assuming the low performance is the fault ML. I would think instead that this low performance is deliberate because it locally optimizes Facebooks KPIs. My understanding is that Facebook has a lot of data on the elasticity of ad buyers. If Facebook were to have even more precise algorithms, then they would likely charge even more money for impressions and clicks. Presumably buyers may be…

If you look at the communication, it seems that the Ad people didn't have a clear idea how well-targeted anything was.

It's easy to see how the statistical problems compound.

FB does not have exact earning levels, so you have to infer that from, say, likes. Let's say you can build a model salary(likeBMW, likeTravel, ...) = %likeBMW + %likeTravel +...

This gives you an estimate which is (70-80)% accurate, so you predict >£250k/yr 70pc of the time. In c. 25% you mispredict.

Now it seems to me that this 25% is going to compound across several categories: when you say "College AND HighEarner AND ..." you get more than 50% of your target group not matching this exact criteria (all you need to fail is one condition to fail to match this conjunction).

And according to FB comms, it looks like >50% didn't match client's chosen criteria.

I think this is the right analysis of the issue. ML systems of this kind are very bad at making targeted judgements. It's really little more accurate than guessing the mean of something (eg., salary) for your group.

All ML has to do, for FB/Google/etc. is improve targeting a few percent to have a significant value proposition.

However, the propaganda has it that ML systems can "target" you, etc. Only in the way a nuclear bomb "targets" a house.

Re: Facebook managers trash their own ad targeting in unsealed remarks

#123

Post by Rastonbury »

mjburgess wrote: post #5

This shows the limitations of ML: facebook has incredible amounts of preference & behaviour data on its users; and can't even meet incredibly generic categories such as "high-earner, college educated, etc.". The reason we think we're in an "AI" boom is 90% these ad. companies hyping their own abilities (an identical strategy to that of the initial boom in the 50s). What we call "AI" today is just an associative house…

I think what's scary is how much a blackbox it is. Some people rave at how the FB algo has improved to the point where they set the broadest ad audience possible. 2 years ago segmentation and testing was the name of the game. Now you apparently can just let it loose

Re: Facebook managers trash their own ad targeting in unsealed remarks

#124

Post by drzoltar »

mjburgess wrote: post #121
drzoltar wrote: post #45

Earlier quoted context omitted.

I’m a bit confused why we are assuming the low performance is the fault ML. I would think instead that this low performance is deliberate because it locally optimizes Facebooks KPIs. My understanding is that Facebook has a lot of data on the elasticity of ad buyers. If Facebook were to have even more precise algorithms, then they would likely charge even more money for impressions and clicks. Presumably buyers may be…

If you look at the communication, it seems that the Ad people didn't have a clear idea how well-targeted anything was. It's easy to see how the statistical problems compound. FB does not have exact earning levels, so you have to infer that from, say, likes. Let's say you can build a model salary(likeBMW, likeTravel, ...) = %likeBMW + %likeTravel +... This gives you an estimate which is (70-80)% accurate, so you predi…

Could it be a misunderstanding of the outcome, or perhaps bias in Facebooks algorithms? The best predictor of your target is: your target. In this case it’s predicting who will buy your product. Using a proxy such as income and education is a good start but surely cannot be the best approach because it artificially limits the search space. The best target is some ML-determined combination of features. My guess is that for niche items such as expensive watches and cars, Facebook’s targeting may be too liberal with lower income earners. But I think this the result of being very good at targeting the general audience for medium-low priced goods and services.

Re: Facebook managers trash their own ad targeting in unsealed remarks

#125

Post by gonehome »

tsimionescu wrote: post #119
gonehome wrote: post #105

Earlier quoted context omitted.

A couple quick things I can think of: - Voice transcription - Tesla Autopilot - Facial recognition (photo sorting on iphones, better photos) - Better graphical performance on Nvidia cards ( https://developer.nvidia.com/dlss ), also better compression for streaming. - Much better translation - Colorizing and repairing old photos - Visual recognition allowing better search of images I’m sure there are some I left out.…

Tesla autopilot is notoriously unsafe, and still extremely bad even at simple problems. Not sure how it could be called a "solved problem" - especially when you have the CEO of Waymo saying publically that he doesn't believe we'll "ever" reach level 5 autonomous driving. There have been massive improvements in automated driving, but if you want to talk about solved problems, parking assisst is as far as you can get.…

Autopilot is distinct from full self driving - I was talking about driver assist.

'Solved' is doing a lot of work here and is an unnecessary threshold I'm willing to concede. I think we would be more likely to agree that things went from unusably bad or impossible to usably good, but imperfect in a lot of consumer categories in the last five years due to ML and deep learning approaches.

The more clearly 'solved' cases of previously open problems (Go, protein folding, facial recognition, etc.) are mostly not in the consumer space.

As far as the gpt-3 bit I encourage others to read that excerpt, they explicitly state they excluded the problems they asked from the test data so it's not memorization. The types of failures it makes are failures like failing to carry the one, it certainly seems like it's deducing the rules. It'll be interesting to see what happens in gpt-4 as they continue to scale it up.

Re: Facebook managers trash their own ad targeting in unsealed remarks

#126

Post by detaro »

gonehome wrote: post #105
nopriorarrests wrote: post #99

Earlier quoted context omitted.

Honest question, no snark --- which consumer space problems were solved, if I don't play Go and don't have FB account to recognize me on a group photos (both of these two statements are true)?

A couple quick things I can think of: - Voice transcription - Tesla Autopilot - Facial recognition (photo sorting on iphones, better photos) - Better graphical performance on Nvidia cards ( https://developer.nvidia.com/dlss ), also better compression for streaming. - Much better translation - Colorizing and repairing old photos - Visual recognition allowing better search of images I’m sure there are some I left out.…

I keep hearing voice transcription is "solved", but both where I see it in consumer products (e.g. YT auto subtitles) and in dedicated transcription services I've tried it's far from solved.

Even the good translation services will randomly produce garbage, struggle with content that's not in nicely fully formed sentences, and are severely limited in language support.

Re: Facebook managers trash their own ad targeting in unsealed remarks

#127

Post by matkoniecz »

amelius wrote: post #92
matkoniecz wrote: post #85

Earlier quoted context omitted.

> I don't care about driving. It still takes the same amount of time. If it would be automated, the you would be able to use this time for something else rather than driving.

In theory yes, in practice mostly no.

I am succesfull with productive and semi-productive activities in a buses (at least when travel time is below 8 hours).

Re: Facebook managers trash their own ad targeting in unsealed remarks

#128

Post by disgruntledphd2 »

fairity wrote: post #79
stonecraftwolf wrote: post #70

Earlier quoted context omitted.

> Furthermore, granular targeting is no longer the way to scale on Facebook. Can you elaborate on this? I’m not sure I understand what the alternative is.

3-5 years ago, the most effective way to design campaigns was by micro-targeting audiences. That is, you would have thousands of adsets, and each adset would target a small slice of your target audience. The theory behind micro-targeting was that your audience's conversion rate varies considerably by certain key targeting attributes. And, if your audience has varying conversion rates, your CPC bids ought to reflect t…

This has actually been true since at least 2013, as long as you were optimising for something that made you money.

If you're optimising for a proxy objective, more granular targeting can make sense (if you have more information about your customers), but if you can just make money from website conversions or in-app purchases, you're better off letting the algorithm do its thing.

(The magic is driven not by incredible engineers but by more people to show the ad to enabling the only people who see it to have high expected conversion rates. This is perhaps the real dirty secret of internet advertising, in that they are predominantly platforms for showing ads to people who were probably going to convert anyway).

Re: Facebook managers trash their own ad targeting in unsealed remarks

#129

Post by disgruntledphd2 »

JumpCrisscross wrote: post #28
rattray wrote: post #19

Earlier quoted context omitted.

What could explain the mechanics of how this happens? Surely FB employees aren't managing those bot accounts themselves. Do they generally determine accounts that engage with ads to be "human/real", thereby incentivizing bad actors who spin up fake FB accounts to click/"like" ads?

> Surely FB employees aren't managing those bot accounts themselves Why not? It's not like Facebook employees have a track record as paragons of virtue. If there are managers incentivized to meet click targets, it could even happen informally. Like the Wells Fargo account-opening scandal.

No business as successful as Facebook is going to do something as idiotic as introducing click targets.

Clearly you would want to optimise the rates ;)

Re: Facebook managers trash their own ad targeting in unsealed remarks

#130

Post by disgruntledphd2 »

was_boring wrote: post #3

Let’s be real, ad tech is a shady business with Facebook leading the charge. This shouldn’t be surprising to any ad executive whose performance isn’t measured by actual dollars generated, but by impressions, money spent, cost per click, and various other dubious metrics which are conveniently provided by the ad network. Not to saying the ad network isn’t valueable (it is and the largest tech companies are ad networks…

You should probably be talking about ad tech apart from Facebook, as very few ad-tech companies have large teams dedicated to help advertisers run experiments on Facebook.

Up till very recently, said teams had a separate reporting line from sales to ensure that they were telling advertisers the truth.

Post reply on HN