Creepy Crawlies

Post by Demiurge » Sun, Aug 30, 2026, 5:22 PM UTC

1oooqooq wrote: post #148

TZubiri wrote: post #111

Earlier quoted context omitted.

There's a winner takes all (monopoly) dynamic, if everyone uses your product, you can share the ip banlists between customers.

See, this is you falling for their marketing. It makes sense doesn't it? If you own all the pipes, you can effectively block bad actors. But in reality what they do is to just block everyone who blocks advertising tracking and profiling tech (such as canvas fingerprint etc)... and because all vendors use cf, they all block those same clients, and they think there's some winner takes all magic happening with finely tu…

Post by tptacek » Sun, Aug 30, 2026, 5:25 PM UTC

harshreality wrote: post #150

tptacek wrote: post #145

Earlier quoted context omitted.

I'm not moralizing. The sites running Anubis are owned by people who like Anubis and are fully within their rights to run Anubis. I don't care how enthusiastic they are about it. It's their call. I'm just saying, it's bad computer science.

You're arguing worst-case, and people running anubis are saying it dramatically helps in their experience, because they're not getting too many of the sorts of bots that people like you and Tavis (correctly) argue are theoretically possible. So the argument that it's "bad CS" falls flat. A better, perhaps the best, argument is that a worse bot mix might appear tomorrow, so anubis should be used as a runway for a bett…

Post by semiquaver » Sun, Aug 30, 2026, 5:26 PM UTC

nneonneo wrote: post #59

tptacek wrote: post #44

Tavis Ormandy called this, about Anubis, almost exactly a year ago: https://news.ycombinator.com/item?id=44962529 It never really cohered as a solution. High-powered scrapers are better equipped to handle proof-of-work challenges than end users. Proof of work makes sense for a password hash, where any one guess at a password provides zero marginal utility. But every request from a scraper is productive to the scraper…

I disagree. The kernel finds it effective - 66% of scrapers are turned away directly. The scraper problem now is fleets of residential proxy devices - often things like smart TVs, phones, and browsers with some “proxy SDK” installed as part of an app’s monetization scheme. They make a couple of requests to a site - just enough to fly under the radar - and move on to a different site. If each new site they hit forces…

  > meaningful dent

Post by paytonjjones » Sun, Aug 30, 2026, 5:28 PM UTC

semiquaver wrote: post #112

wongarsu wrote: post #100

Earlier quoted context omitted.

Isn't the issue here the three order of magnitude difference between the (presumably optimized) JS implementation and the optimized C kernel on your browser? If the two stay within an order of magnitude of each other Anubis is a perfectly viable and scalable solution. Bots want to open many orders of magnitude more pages than a normal user, so the resource investment for each single page matters a lot more to them Fo…

> I really don't see why Safari on a modern iPhone should be so slow at this me neither, but I don't think it changes the argument. There's always going to be someone on a low-end device. Your adversaries already have superhuman coding ability and infinite patience. Why would you expect the long-term advantage to be with the defenders?

Post by alwa » Sun, Aug 30, 2026, 5:30 PM UTC

voakbasda wrote: post #71

NooneAtAll3 wrote: post #57

from what I see there are 2 solutions: 1) ban TV-proxy-as-a-service - straight up go to every representative there is and start pushing and lobbying and everything to stop spammers from distributing over non-computer devices, especially legally 2) make old commits more expensive to access than new ones. Legit users are not going to access those much, so they can pay the time. I assume diverse (unpredictable?) difficu…

I would think that residential proxying would be illegal already, as it is a network intrusion. The trick is chasing down the offenders, proving their actions did harm, and getting them to pay. None of those steps are easy, even if there are laws to assist. Otherwise, spam would be a solved problem.