Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare's new marketplace lets websites charge AI bots for scraping

techcrunch.com

Post reply on HN
Print view View original

191–200 of 280 posts

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#191

Post by AyyEye »

gruez wrote: post #188
AyyEye wrote: post #183

Earlier quoted context omitted.

1. A blacklist isn't profiling. Known problem causing entities is entirely different than 'he looks suspicious', because the latter is often... Misused (to be polite). 2. Of course it is relevant. Because the more false positives they have the more money they can extort. They have negative incentive for their system to work properly. P.S. ticketmaster is absolutely criminal, too.

>2. Of course it is relevant. Because the more false positives they have the more money they can extort. They have negative incentive for their system to work properly. What are the "false positives" in this context? It's specifically for blocking bots, and enrollment into the program to get unblocked is designed for bot owners. It's obviously not designed to extract money from regular users. I doubt there's even a s…

> It's specifically for blocking bots

Define "bots" in a way computers can understand.

> What are the "false positives" in this context?

Regular users that cloudflare (profiles) accuses of being bots. God help you if you want to block trackers or something else that's not regular.

> I doubt there's even a straightforward way for regular users to pay to get unblocked via this channel

This is part of the problem. But hey, at least they are only a process change away from charging normies too.

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#192

Post by koito17 »

rmbyrro wrote: post #190
Aachen wrote: post #83

Earlier quoted context omitted.

> gatekeep access to those who pay and those who don't, and that applied whether they are bots or people. I'm already constantly being classified as bot. Just today: To check if something is included in a subscription that we already pay for, I opened some product page on the Microsoft website this morning. Full-page error: "We are currently experiencing high demand. Please try again later." It's static content but i…

If you use Linux, the experience is terrible nowadays. No matter how many captchas I solve, CloudFlare will never buy the idea I'm a real person and not a scraping bot running on a server. I wonder if this kind of discrimination is even legal...

Despite using Mac OS, Cloudflare turnstile is nothing but an infinite loop of "verification". I am using Firefox with basic privacy protections enabled. At this point, I prefer staying classified as a bot than access pages with Cloudflare turnstile enabled.

Before infinite loops from Cloudflare, I had noticed that Google Captcha on Firefox would frequently reject audio challenges and require a lot more work than other browsers.

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#193

Post by n_ary »

kylehotchkiss wrote: post #175

Is anybody else seeing an absolutely massive amount of Amazonbot crawls on their site? What are they up to? And why so aggressively?

Most likely aspiring AI startups gathering as much data as they can before regulation jaws snap shut around them cutting off the blood stream.

In this AI race(hype), data is finally the ultimate gold. Also at the rate the information is polluted by GenAI junk all over, any remnants of real data is holy grail.

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#195

Post by ForOldHack »

klyrs wrote: post #166
diggan wrote: post #130

Earlier quoted context omitted.

> Those "keep clicking until we stop fading in more results" challenges mean they're fairly confident you're a bot Those ones are the fucking worst. I've noticed that if I try to succeed in these captchas too quickly, it'll just say "Sorry, try again" even when every click was correct, so instead, I've started going in slow motion and faking "misclicking" which makes it much more likely to accept me as human. I canno…

I always spoil as many of these as possible. Sometimes it takes me a while to prove that I'm human, but I'm dead-set on convincing it that I'm a stupid human. Of course, I fantasize that some day a robo-car will crash because I taught it that there's really no difference between a motorcycle and a flight of stairs.

I was waiting for the day that two SUVs would hit each other, and I happened.

Now I am waiting for two self driving cars to hit each other... they already drive like "American idiots", guess we know what the training model is.

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#197

Post by ForOldHack »

shadowgovt wrote: post #150
Aachen wrote: post #102

Earlier quoted context omitted.

Those "keep clicking until we stop fading in more results" challenges mean they're fairly confident you're a bot and this is the highest difficulty level to prove your lack of guilt. I get these only when using a browser that isn't already full of advertising cookies (edit: which, to be clear, I hope is still considered an acceptable state to have your browser in)

It's acceptable, but suspicious. Two standard deviations away from the median browser (and a lot more like the configuration of a scraper, which would get reloaded in some Docker instance frequently with a fresh empty cookie jar because storing data costs infrastructure).

You mean Edge? Chrome stands a 65.2% ( 1 deviation ) Safari at 18.57% ( 2 deviations ), so Edge at 5.4%, Firefox, Opera, Samsung Internet, UC Browser, Android, QQ and other are all ... deviants?

https://gs.statcounter.com/browser-market-share

I use Firefox nightly which does not even show up statistically...

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#198

Post by gruez »

AyyEye wrote: post #191
gruez wrote: post #188

Earlier quoted context omitted.

>2. Of course it is relevant. Because the more false positives they have the more money they can extort. They have negative incentive for their system to work properly. What are the "false positives" in this context? It's specifically for blocking bots, and enrollment into the program to get unblocked is designed for bot owners. It's obviously not designed to extract money from regular users. I doubt there's even a s…

> It's specifically for blocking bots Define "bots" in a way computers can understand. > What are the "false positives" in this context? Regular users that cloudflare (profiles) accuses of being bots. God help you if you want to block trackers or something else that's not regular. > I doubt there's even a straightforward way for regular users to pay to get unblocked via this channel This is part of the problem. But h…

>Define "bots" in a way computers can understand.

How is having a specific definition relevant to this conversation? An approximate definition of "a human using a browser to visit a site" probably suffices, without having to get into weird edge cases like "but what if they programmed lynx to visit your site at 3am when they're asleep?".

>Regular users that cloudflare (profiles) accuses of being bots. God help you if you want to block trackers or something else that's not regular.

I use ublock, resistfingerpnting, and a VPN. That probably puts me in the 95+ percentile in terms of suspiciousness. Yet the most hassle I get from cloudflare is the turnstile challenges can be solved by clicking a checkbox. Suggesting that this sort of a hurdle constitutes some sort of "criminal enterprise" is laughable.

I do occasionally get outright blocked, but I suspect that's due to the site operator blocking VPN/datacenter ASNs rather than something on cloudflare's part.

>This is part of the problem. But hey, at least they are only a process change away from charging normies too.

So they're damned if they do, damned if they do? God forbid that site operators have agency over what visitors they allow on their sites!

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#199

Post by Terr_ »

theyeenzbeanz wrote: post #98
Aachen wrote: post #83

Earlier quoted context omitted.

> gatekeep access to those who pay and those who don't, and that applied whether they are bots or people. I'm already constantly being classified as bot. Just today: To check if something is included in a subscription that we already pay for, I opened some product page on the Microsoft website this morning. Full-page error: "We are currently experiencing high demand. Please try again later." It's static content but i…

Lately I’ve been noticing captchas have been increasingly difficult day by day on Firefox. Checking the box use to go through without issue, but now it’s been starting to pop up challenges with the boxes that fade after clicking. Just like your experience, chrome has no hiccups on the same machine.

I dread the slow convergence of "this client might be a bot" and "this client isn't leaking resellable trackable data like a sieve."

Re: Cloudflare's new marketplace lets websites charge AI bots for scraping

#200

Post by troyvit »

johnklos wrote: post #124
troyvit wrote: post #42

Earlier quoted context omitted.

As an actual content provider I see this as an opportunity. We pay our journalists real money to write real stories. If AI results haven't started affecting our search traffic they will start to soon. Up until now we've had two choices: block AI-based crawlers and fall completely out of that market, or continue to let AI companies train off of our hard-won content and take it as a loss that still generates a little b…

> don't blame the player, blame the game You make it sound like this is OK. "It's not their fault that a protection racket didn't already exist. They just filled the market's need for one."

I do hate it whenever somebody says that line to me, because it's up to the player to choose if they want to play, and that automatically puts them in a certain bucket.

I believe the game is rigged from the get-go. Nobody should be able to get that big without having a level of accountability that matches their size, and our current economic system doesn't support that. That's why X can go one way with content moderation, Meta another, etc. and whole countries get pissed off. That's why I hate the game. The players have scaled past it.

Web infrastructure is headed in that direction more and more too. I personally think that for all their reach and influence Cloudflare does a great job protecting the internet, but that can change at any time and it would be in nobody's control but Cloudflare's. For now I'm glad it's them and not AWS or Alphabet. I don't know how I'll feel in five years.

Post reply on HN