Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare Turnstile requiring fingerprintable WebGL

hacktivis.me

Post reply on HN
Print view View original

291–300 of 508 posts

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#291

Post by danielheath »

userbinator wrote: post #276
danielheath wrote: post #245

Earlier quoted context omitted.

They showed up when the AI money did. The evidence is circumstantial, but… some of them are remarkably well engineered (from a “how difficult is it to identify this traffic” perspective, in a way that never existed before (I have been running a quite sizeable site for 8 years, over 200k registered users, and you don’t need to register to use 99% of it).

Yes, circumstantial is exactly the point; it's easy to use AI as a scapegoat because it's something popular to hate on.

It's circumstantial evidence, but Occam's Razor also applies.

It's not a hostile DOS in the traditional sense (I've mitigated a few of those) - no "pay us to make it stop", no pattern to the requests other than "fetch every unique URL a few times".

It wasn't happening until financial incentives to gather large datasets for AI training appeared.

Bad actors (using residential proxies & claiming to be a real browser) mostly showed up after folk started blocking ones that identified themselves as AI scrapers.

It's obvious to blame AI training because there's a shortage of better explanations. Who else would be paying for these (expensive) residential botnets, only to use them to (eg) web-scrape wikipedia (which offers free downloads of its content in a structured format)?

The simplest explanation of the technical behavior is "a bot coded to follow every link it sees & save the results", and the simplest explanation of the motive to run such a bot is "to train a large language model".

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#292

Post by adgjlsfhk1 »

ethin wrote: post #164
jwr wrote: post #96

Earlier quoted context omitted.

> I don't want to defend them, because they gate away a good chunk of the internet with their "bot protection" They also gate away a good many people with their "bot protection". I am extremely worried about how so many seem to have outsourced the control over who can access their websites to a company, with no second thoughts whatsoever.

The problem is what is the alternative? I'm (not) defending them or this practice by any measure, but we all know what happens if you just open your site up without these, especially with AI bots which hammer servers and are in effect a legalized DDoS system. I've hated CAPTCHAs ever since I first encountered them and I can't wait for them to just finally die a permanent death, but I also don't know how we solve the…

I think there's some chance we get a "proof of purchase" system where there is some entity that takes a $10 payment to give out a unique identity token that you need to present to visit most sites. if you have a revocation process for ones used for bad actors, it seems like it would work pretty well.

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#293

Post by hombre_fatal »

jwr wrote: post #165
ethin wrote: post #164

Earlier quoted context omitted.

The problem is what is the alternative? I'm (not) defending them or this practice by any measure, but we all know what happens if you just open your site up without these, especially with AI bots which hammer servers and are in effect a legalized DDoS system. I've hated CAPTCHAs ever since I first encountered them and I can't wait for them to just finally die a permanent death, but I also don't know how we solve the…

> we all know what happens if you just open your site up without these, especially with AI bots which hammer servers and are in effect a legalized DDoS system No, we don't know. I honestly do not understand the problem. I run websites, both static and non-static. Granted, my sites aren't exactly the most popular internet go-to destinations, but I should be seeing this DDoS too, right? I do see lots of requests. Nothi…

Consider yourself lucky. But don't let yourself fall into the trap of thinking it's a nonissue for everyone else until it happens to you.

People shouldn't have to be experts or provision a larger server to run a UGC service that can withstand the sort of 30x more traffic I'm seeing from AI bots. Or rather, you didn't render the argument for why they should have to do that if they can just use CloudFlare's free tier.

Either way, it's easy to have all the answers when you've never had the problem.

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#294

Post by bawolff »

denysvitali wrote: post #11

Cloudflare is known to use fingerprinting to detect scrapers For example, they use JA3 fingerprints and match them against the UA to block stuff like cURL while allowing OkHttp (Android clients) - but this can be easily be spoofed with packages such as CycleTLS [1]. I don't want to defend them, because they gate away a good chunk of the internet with their "bot protection", but unless you do PoW (which is also ecolog…

> I don't want to defend them, because they gate away a good chunk of the internet with their "bot protection", but unless you do PoW (which is also ecologically a nightmare), probably fingerprinting is the way to go - completely destroying the privacy of everyone involved.

I hate what the anti scrapper mechanisms have become but it really is the lesser evil. The alternative for many small operators is to just completely shutdown.

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#295

Post by dcrazy »

dotancohen wrote: post #214
dcrazy wrote: post #106

Earlier quoted context omitted.

Because it doesn’t solve the problem of residential botnets.

Neither does fingerprinting.

The goal of Cloudflare’s fingerprinting is to detect whether a user agent appears to be a legitimate human. It’s not to identify human users across websites.

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#296

Post by whstl »

danielheath wrote: post #245
userbinator wrote: post #180

Earlier quoted context omitted.

Also, how do we even know they're really "AI scrapers", or just a deliberate DDoS to push sites into using CF or other "anti-bot" providers?

They showed up when the AI money did. The evidence is circumstantial, but… some of them are remarkably well engineered (from a “how difficult is it to identify this traffic” perspective, in a way that never existed before (I have been running a quite sizeable site for 8 years, over 200k registered users, and you don’t need to register to use 99% of it).

I run a quite large website and there are a few patterns.

The usage is extremely quick, and follows easy-to-spot patterns. We noticed a spike in bounce rate.

They never come from Google, and the bad programmed ones just crawl several pages at a time, faster than a user could do.

Then there's the crazy spikes in visits from specific countries, pretty much scraping the entire content. Often from pools of IPs. In some cases had 30% unexplained (meaning: it wasn't viral or a marketing campaign) random sustained increases in traffic.

There's also the fact they don't interact with the complicated widgets, so zero XHR requests other than analytics pings.

They also don't cause spikes in Google Analytics, so I assume it's blocked, but they show up in logs and in the internal analytics.

It's not enough to DDOS the website at all, but it's a lot of noise in statistics that we gotta learn to filter.

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#297

Post by zzo38computer »

dcrazy wrote: post #295
dotancohen wrote: post #214

Earlier quoted context omitted.

Neither does fingerprinting.

The goal of Cloudflare’s fingerprinting is to detect whether a user agent appears to be a legitimate human. It’s not to identify human users across websites.

That is not a good excuse for requiring overly complicated and overly specific software.

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#298

Post by black_puppydog »

wizzwizz4 wrote: post #268
ceejayoz wrote: post #255

Earlier quoted context omitted.

It’s not just scale. Do you accept user generated content? If so, more of a target.

Stack Overflow was outside of the Cloudflare network for years , and anti-abuse was maybe 3 or 4 full-time jobs – much of which still needs to be done, because Cloudflare's anti-bot protection hasn't actually stopped it. Most UGC sites are not as big as Stack Overflow was at its peak.

errr... so anything related to UGC now has a lower bound of 3-4 FTE? Sure, I'll hire a team of content moderators next time I think about putting a comment form under my blog...

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#299

Post by epistasis »

dotancohen wrote: post #211
CrimsonRain wrote: post #160

Earlier quoted context omitted.

I'm one of those who have enabled cloudflare on all of the sites I maintain. Additionally, Added turnstile on every form. I know some actual users get blocked. But the amount of spam we get without it, the amount of bot traffic simply overwhelming the server... It is just too much. Recently I also hard blocked all IPs from china Singapore India Pakistan Russia and whole of africa. Do I want to do it? No. But the amou…

> I know some actual users get blocked. But the amount of spam we get without it, the amount of bot traffic simply overwhelming the server... It is just too much. So why not just shut down the website? Or remove the form entirely? That will ensure that you get no spam, right? One of the core tenets of system design is Availability. If your service is not available - if your forms are blocking legitimate users - then…

> So why not just shut down the website? Or remove the form entirely? That will ensure that you get no spam, right?

Turns out that people have a tolerance for a non-zero amount of work, but still have a limit.

Suggesting "turn off your website" is does not account for the desire to also provide some access.

Treat people who host content as humans, just as we must treat users as humans. There are tradeoffs, suggesting "shut down your website unless you provide access everywhere" is worse on all fronts for everyone.

Re: Cloudflare Turnstile requiring fingerprintable WebGL

#300

Post by aboardRat4 »

CrimsonRain wrote: post #160
binaryturtle wrote: post #141

Earlier quoted context omitted.

I can no longer access any website that's "protected" by Cloudflare. As soon a website enables that stuff… "Shoot, another one bites the dust." I wonder if the website owners realise at all how many actual users they lose by this sort of "protection."

I'm one of those who have enabled cloudflare on all of the sites I maintain. Additionally, Added turnstile on every form. I know some actual users get blocked. But the amount of spam we get without it, the amount of bot traffic simply overwhelming the server... It is just too much. Recently I also hard blocked all IPs from china Singapore India Pakistan Russia and whole of africa. Do I want to do it? No. But the amou…

[flagged]
Post reply on HN