Copying the text just incase FB decides they don't like it --- Hey everyone, we're going to be deleting our Facebook page in the next couple of weeks, but we wanted to explain why before we do. A couple months ago, when we were preparing to launch the new Limited Run, we started to experiment with Facebook ads. Unfortunately, while testing their ad system, we noticed some very strange things. Facebook was charging us…
And here is a screenshot: http://i.imgur.com/4hu66.png
Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
191–200 of 381 posts
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#192Earlier quoted context omitted.
In my mind having an in-house bot network and being totally ignorant that you're charging for bot traffic via ads amount to the same thing. This is very, very bad.
How? In house bot network implies active fraudulent charging, while being unaware of bot clicks is simply neglect or ignorance.
Yes, we dont need a conspiracy to explain stupidity. The stupidity is the conspiracy.
People arent ignorant, because they are handcapped or something. They are ignorant, because thats the behaviour and excuse we reward. Just be an idiot, dont put in the intellectual effort, and you will be allowed to speak lies and screw every one and every thing.
From a corporate point of view, its the golden ticket. Plausible deniability. We didnt know that our insurance helped finance weapon exports to dictators. We didnt know. We didnt know. Sorry this, sorry that. We're keeping the money though.
I dont think that provable intent should have legal meaning. Because all neglect is intentional, as is ignorance in a world, where every one has the choice to be well informed.
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#193My startup is essentially an advertising aggregator (pooling traffic from a variety of publishers and routing it to advertisers) and dealing with things like bot detection is a HUGE chunk of what we work on, technology-wise. Let me try and give you an idea of how deep the rabbit hole can go. - Okay, you want to detect bots. Well, "good" bots usually have a user agent string like, "Mozilla/5.0 (compatible; Googlebot/2…
And then there are the "wet bots" - Amazon Turks and overseas humans (lowly) paid to use a regular browser and PC and manually click on stuff for whatever purpose (rig internet voting, drive fake advertising charges, steal data, skew hit counters - whatever).
What you are getting at however is very important to ad-driven internet business models. You make the empathetical deduction that if you are there for one purpose, you are impervious to advertising.
Now, if you are still with me after that statement, then there are larger impications of online advertising than are actually being measured. For instance, if the intent of the view mattered, would it be worth more? I think Google has answered that question rather clearly.
Measuring "why someone is visiting a URL" (and thus generating an ad-view) is a larger technical problem than could be accomplished with some fancy algorithms. That is, unless the service knew all of your browsing history...
In any case, you raise a good point about the intent of the page-view and how that should be "credited" as advertising. To do this however is to challenge existing 'fundamental' theories.
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#194Earlier quoted context omitted.
It is not my responsibility to educate you on modern internet history. This has been argued here on HN prior; Do you recall the backdoors that AT&T put at 600 Folsom, in SF? And the backdoors in Cisco's equipment that are a requirement by the federal government? Maybe you thin this shit is "conspiracy theory" and if you do, you're simply a naive fool. Look around you at what governments are doing. The NSA has records…
It's odd to me that your posts aren't more downvoted than they are.
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#195My startup is essentially an advertising aggregator (pooling traffic from a variety of publishers and routing it to advertisers) and dealing with things like bot detection is a HUGE chunk of what we work on, technology-wise. Let me try and give you an idea of how deep the rabbit hole can go. - Okay, you want to detect bots. Well, "good" bots usually have a user agent string like, "Mozilla/5.0 (compatible; Googlebot/2…
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#196My startup is essentially an advertising aggregator (pooling traffic from a variety of publishers and routing it to advertisers) and dealing with things like bot detection is a HUGE chunk of what we work on, technology-wise. Let me try and give you an idea of how deep the rabbit hole can go. - Okay, you want to detect bots. Well, "good" bots usually have a user agent string like, "Mozilla/5.0 (compatible; Googlebot/2…
As in your case, it's nearly impossible to detect click fault real time as the clicks streaming in. We batch-processed the clicks throughout the day and do daily billing settlement at the end of the day. It's easier to detect click fault over a period of time. Obvious bots are filtered out, like the search engine crawlers.
For click fault, it's impossible to distinguish one-off click as a user click or fault click. We tried to catch repeated offenders. we looked for duplicate clicks (same ip, same agent, other client attributes) against certain targets (same product, product categories, merchants, etc) over a period of times (1-minute, 5-minute, 30-minute, hours, etc). We also built historic click stats on the targets so we can check the average click pattern against historic stats (yesterday, week ago, month ago, year ago).
We ran quick checks every 1 minute and 5 minutes to detect unusual click patterns. These were mainly for detecting DOS attack against our site or against certain products. We actually had a realtime reaction mechanism to deal with it: slow down the requests, throw up CAPTCHA, redirect to a generic page, or block ip at routers.
Every couple hours, we batch processed the clicks so far and ran the exhausting checks to flag suspicious clicks. At the end of the day, definite fault clicks were thrown out, moderate suspicious clicks were grouped and reviewed by human. It's a fine balance to flag less suspicious clicks since there's money involved.
The sophisticate click fault I've seen involved a distributed botnet clicking from hundreds of IP all over the world doing slow clicks during the day. We found out because it's targeting a product category of a merchant and causing unusual click stats compared to historic pattern.
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#197Earlier quoted context omitted.
How would you use the access log to determine if client-side javascript was being executed?
Check if it's even being fetched for a start. Assuming it's not inline.
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#198My startup is essentially an advertising aggregator (pooling traffic from a variety of publishers and routing it to advertisers) and dealing with things like bot detection is a HUGE chunk of what we work on, technology-wise. Let me try and give you an idea of how deep the rabbit hole can go. - Okay, you want to detect bots. Well, "good" bots usually have a user agent string like, "Mozilla/5.0 (compatible; Googlebot/2…
Instead of that, we'll watch for patterns of use in which high volumes of traffic come from unrecognized non-end-user network space.
"Unrecognized" means that it's not one of your usual search vendors: Google, Yahoo, Bing, Baidu, Yandex, Bender, AddThis, Facebook, etc.
"Non-end-user network space" means a netblock which is clearly not a residential or commercial/business networking provider.
For this you'll mostly want to eyeball, though when we saw very high volumes of search coming at us from ovh.net (a French low-cost hosting provider), it was pretty obvious what the problem was.
Using an IP to ASN/CIDR lookup (such as the Routeviews project: http://www.routeviews.org/) and a local ASN -> name lookup. One source of this is http://bgp.potaroo.net/cidr/autnums.html
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#199Earlier quoted context omitted.
> (or that hit the page and left it before the js executed, which seems improbable for such a large number - 70%). That's actually an interesting proposal-- 100% of the time when I click on an ad, I didn't mean to and I try to close it before it has a chance to annoy me. Probably doesn't make sense on a large scale, but it's something to consider.
I agree that some users might close the page or hit the "back" button before the $(document).ready() happens, but I can't believe that 70% of the people did that(taking into account that our ad was really targeted and highly relevant to the landing page).
Re: Company withdrawing from Facebook as analytics show 80% of ad clicks from bots
#200Some background and a disclaimer. First the disclaimer, I've worked at Google (which lives of its search advertising) and now I'm with Blekko (which generates revenue from advertisements as well) so I've been on the 'inside' looking out at this problem. The background. A lot of the web you see today is automated, which is to say that 'the web' can exert influence in the 'real' world, and influencing the web can be am…
This just begs the question -- how difficult is it to do a reasonable job of detecting bots before passing through the "click"? If it is really hard to do during the request processing, surely it is not hard to do before billing?