Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bots are getting good at mimicking engagement

joindatacops.com

Post reply on HN
Print view View original

21–30 of 312 posts

Re: Bots are getting good at mimicking engagement

#21

Post by lazide »

nemomarx wrote: post #19
chrismorgan wrote: post #17

I’m puzzled by this: I thought it was well-understood, at least in the industry, that traffic numbers were at least mostly nonsense, and that ad click metrics especially were suuuper shady, typically more than half fraud; yet OP, in the business of “accurate ad spend analytics”, only just discovered this!? It just doesn’t ring true. That aspect of the story isn’t novel at all, and someone in that line of work should…

50k to 47 conversions seems like a noticeable increase in fakeness to me at least. that's going from more than half fraud to essentially only fraud with a rounding error amount of real users

This is the ‘gold’ that agentic/generative AI really produces.

For most consumers, it’s entertainment, but for industrial use it’s great for this kind of fraud. And very difficult to detect at scale, since the only cost effective tools for this kind of analysis are also ML/AI, and hence can be fooled more predictably/trained against.

Re: Bots are getting good at mimicking engagement

#22

Post by nurettin »

> scraping 70 million retailer web pages every single day. This is a legitimate and massive source of automated traffic.

Why do they do this? For vital business intelligence. Major retailers like Amazon do not always notify vendors when they run out of stock. So, brands pay for data scraping services to monitor their own products. These "good bots" check inventory levels, see who is winning the "buy box," ensure product descriptions are correct, and track search result rankings. They even scrape from different locations and mobile device profiles to analyze what banner ads are being shown to different audiences.

_---------------_

Guilty as charged. You quickly learn to bypass bot detection measures and create a fully automated system to gather all this information just because amazon doesn't provide it in an accessible manner causing harm to businesses who need this intel and their own internet infra.

Re: Bots are getting good at mimicking engagement

#23

Post by viraptor »

nemomarx wrote: post #20
viraptor wrote: post #14

Earlier quoted context omitted.

Yeah, their list of recommendations could use another point: expose the public data in a simple, structured way. I'm working right now on an inventory management system for a clinic which would really benefit from pulling the prices and availability from a very specialised online shop. I wish I could just get a large, fully cached status of all items in a json/CSV/whatever format. But they're extremely not interested…

I wonder if LLM agents will know to go for apis and data or if they'll keep naively scraping in the future. A lot of traffic could come down to "find me x product online" chats eventually

https://llmstxt.org/ is there for that purpose.

Re: Bots are getting good at mimicking engagement

#24

Post by bdcravens »

viraptor wrote: post #14
sixthDot wrote: post #8

What is really mind blowing is that, if understood correctly, bots would be used to check the availability of a product, that sounds so a "hacky" method, like "seriously people are doing that in 2025".

Yeah, their list of recommendations could use another point: expose the public data in a simple, structured way. I'm working right now on an inventory management system for a clinic which would really benefit from pulling the prices and availability from a very specialised online shop. I wish I could just get a large, fully cached status of all items in a json/CSV/whatever format. But they're extremely not interested…

HTML is the only truly universal standard.

Re: Bots are getting good at mimicking engagement

#25

Post by Closi »

viraptor wrote: post #14
sixthDot wrote: post #8

What is really mind blowing is that, if understood correctly, bots would be used to check the availability of a product, that sounds so a "hacky" method, like "seriously people are doing that in 2025".

Yeah, their list of recommendations could use another point: expose the public data in a simple, structured way. I'm working right now on an inventory management system for a clinic which would really benefit from pulling the prices and availability from a very specialised online shop. I wish I could just get a large, fully cached status of all items in a json/CSV/whatever format. But they're extremely not interested…

The counterpoint is 'Why hand your competitors data on a silver plate'?

Sure you might be willing to build the bot to scrape it... but some other competitors won't go to this effort so it still means a bit of information asymmetry and stops some of your competitors poaching customers / employing various marketing tactics to exploit short term shortages or pricing charges etc.

Re: Bots are getting good at mimicking engagement

#27

Post by awongh »

I don't really believe the main thesis of this article- it reads like much of the fake cliff-hanger pseudo-insight endemic to marketing and business influencers.

Mainly, it avoids the main point- 73% of your traffic is "faked" enough to look real.

Who are the players in that scenario that stand to benefit from your traffic being fake?

You pay for Google (search ads) and Facebook ads but the traffic is faked by them (unlikely)

You pay other publishing networks (maybe adsense?) and the website owners profit from sending fake traffic (maybe true? if the article were really trying to make a case for this, just name them?)

Or, you work inside a company and just want to make your department look good?

I'm not sure I know what the point of this article is besides a click bait title.

Just tell me exactly what the mechanism is for this fake traffic- don't hint at some kind of conspiracy.

Re: Bots are getting good at mimicking engagement

#28

Post by yellow_lead »

simul007 wrote: post #2

Hi HN. I run a marketing agency and fell down this rabbit hole after a client's analytics made no sense (50k visitors, 47 sales). I ended up building a simple script to track user behavior and analyzed 200+ small e-commerce sites. The average was 73% bot traffic that standard analytics counts as real. The bots are getting creepily good at mimicking engagement. I wrote up my findings, including some of the bizarre pat…

This is so common, I find it strange to see a whole article framing it as a shocking phenomenon. Like, have you heard of PostHog?

Re: Bots are getting good at mimicking engagement

#30

Post by rightbyte »

simul007 wrote: post #2

Hi HN. I run a marketing agency and fell down this rabbit hole after a client's analytics made no sense (50k visitors, 47 sales). I ended up building a simple script to track user behavior and analyzed 200+ small e-commerce sites. The average was 73% bot traffic that standard analytics counts as real. The bots are getting creepily good at mimicking engagement. I wrote up my findings, including some of the bizarre pat…

What do you think is the false negative ratio of the remaining 27%? A 0.1% total conversion rate would imply 1/270 visitors buying stuff and I am quite certain I buy stuff in maybe every 10th online store I visit or something.
Post reply on HN