Earlier quoted context omitted.
Don’t put your stuff on the internet then, or put it behind a paywall/registration.
What you propose is making the web worse for everyone , instead of a minority of users (AI agents)
Cloudflare's new marketplace lets websites charge AI bots for scraping
181–190 of 280 posts
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#182Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#183Earlier quoted context omitted.
"Security" in the real world doesn't get to profile people. Profiling is Cloudflare's entire business model.
>"Security" in the real world doesn't get to profile people 1. yes they do. have you ever been to vegas? there's cameras and facial recognition everywhere. outside of vegas, some bars and clubs also use ID scanning systems to enforce blacklists, and in most cases that system is outsourced to an external vendor. finally, ticketmaster requires an account to use, and to create an account you need to provide them your bi…
2. Of course it is relevant. Because the more false positives they have the more money they can extort. They have negative incentive for their system to work properly.
P.S. ticketmaster is absolutely criminal, too.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#184Earlier quoted context omitted.
I always spoil as many of these as possible. Sometimes it takes me a while to prove that I'm human, but I'm dead-set on convincing it that I'm a stupid human. Of course, I fantasize that some day a robo-car will crash because I taught it that there's really no difference between a motorcycle and a flight of stairs.
You'll just be lower on the list the AI makes of people that would be a threat.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#185This seems like a gimmick. Isn't preventing crawling a sisyphean task? The only real difference this will make is further entrenching big players who have already crawled a ton of data. And if this feature comes at the cost of false positives and overbearing captchas, it will start to affect users.
My website contains millions of pages. It's not hard to notice the difference between a bot (or network) that wants to access all pages and a regular user.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#186Earlier quoted context omitted.
I mostly agree with you but do find it a fair point to suggest making it a straight-up paywall then. If they want some clients to pay for the content based on heuristic and black-box algorithms, that's going to be discriminatory, we just don't know to which groups (could be users from cheap connections or lower-income countries, could be unusual user agents like Ladybird on macOS, could be anything)
Perhaps, but I’m not sure how different that would be in practice. I have no more idea how the NYT implemented their paywall than Cloudflare does.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#187Boy I'm sick of clicking "Verify you are human" on everything from GitLab to banking apps running Cloudflare. Sick enough that I hope someone prominent at the EFF or similar takes Cloudflare to court over it. One company shouldn't be allowed to police access to the internet. And certainly shouldn't be allowed to start gatekeeping what is viewable by discriminating against the person or software doing the viewing. I w…
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#188Earlier quoted context omitted.
>"Security" in the real world doesn't get to profile people 1. yes they do. have you ever been to vegas? there's cameras and facial recognition everywhere. outside of vegas, some bars and clubs also use ID scanning systems to enforce blacklists, and in most cases that system is outsourced to an external vendor. finally, ticketmaster requires an account to use, and to create an account you need to provide them your bi…
1. A blacklist isn't profiling. Known problem causing entities is entirely different than 'he looks suspicious', because the latter is often... Misused (to be polite). 2. Of course it is relevant. Because the more false positives they have the more money they can extort. They have negative incentive for their system to work properly. P.S. ticketmaster is absolutely criminal, too.
What are the "false positives" in this context? It's specifically for blocking bots, and enrollment into the program to get unblocked is designed for bot owners. It's obviously not designed to extract money from regular users. I doubt there's even a straightforward way for regular users to pay to get unblocked via this channel. As the people who are running blocks and are blocked, I don't see what the issue is. Isn't it working as intended by definition?
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#189AI scrapers are parasites. I don't care whether you're OpenAI, Amazon, Meta, or some unknown startup. As soon as you generate a noticeable load on any of the servers I keep my eyes on, you'll get a blank 403 from all of the servers, permanently. I might allow a few select bots once there is clear evidence that they help bring revenue-generating visitors, like a major search engine does. Until then, if you want traini…
I've been making crawlers for a living! Thanks for informing me that I'm a parasite.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#190Common Crawl is shown in their screen shot of "Providers" along side OpenAI and Antropic. The challenge is that Common Crawl is used for a lot of things that are not AI training. For example, it's a major source of content for the Wayback machine. In fact, that's the entire point of the Common Crawl project. Instead of dozens of companies writing and running their (poorly) designed crawlers and hitting everyone's sit…
> gatekeep access to those who pay and those who don't, and that applied whether they are bots or people. I'm already constantly being classified as bot. Just today: To check if something is included in a subscription that we already pay for, I opened some product page on the Microsoft website this morning. Full-page error: "We are currently experiencing high demand. Please try again later." It's static content but i…
No matter how many captchas I solve, CloudFlare will never buy the idea I'm a real person and not a scraping bot running on a server.
I wonder if this kind of discrimination is even legal...