Earlier quoted context omitted.
I would say let's get rid of copyright and software patents altogether ;)
they're already gone but only if you're well funded (OpenAI)
Remember that open source AI exists.
Cloudflare's new marketplace lets websites charge AI bots for scraping
241–250 of 280 posts
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#242Earlier quoted context omitted.
> The content creator bears all the burden of creating and hosting the content and reaps non of the reward as the AI agent has now inserted itself as a middleman. As a user agent my god what's happened to our industry. Locking the web to known client which are sufficiently not the user's agent betrays everything the web is for. Do you really hate AI so much that you'll give up everything you believe in to see it hurt…
Like I said in another comment, I'm pointing out what is going to actually happen based on incentives, not what I want to happen. I'd much rather the open web continue to exist and I think AI will be a beneficial thing for humanity. edit: to be clear, it's already happening. Blogs are moving to substack, twitter blocks crawling, reddit is going the same way in blocking all crawlers except google.
To be optimistic, as long as anonymous access is a thing, or creating free accounts is a thing, such crawler blocks can probably be bypassed. I hope so, at least.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#243Earlier quoted context omitted.
How is this different than say, ticketmaster charging money to not get "blocked" from a venue (ie. a ticket)?
Because those websites cloudflare is performing racketeering-as-a-service for are open to the public.
> are open to the public
Most websites aren't "open to the public". Most use firewalls, configure rules, etc that already block certain accesses. It's open to selected groups, just maybe including 1s you're allowed to be a part of.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#244Earlier quoted context omitted.
Cloudflare has probably noticed those proxy networks are quite expensive.
Sometimes get hit by the captcha but captcha solvers are cheap (0.3 cents a captcha).
Related article:
"An AI can beat CAPTCHA tests 100 per cent of the time" https://www.shiningscience.com/2024/09/an-ai-can-beat-captch...
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#245Earlier quoted context omitted.
> last I heard of Cloudflare spinning something up just to wind it back down Cloudflare bet big on NFTs ( https://blog.cloudflare.com/cloudflare-stream-now-supports-n... ), Web3 ( https://blog.cloudflare.com/get-started-web3/ ), Proof of stake ( https://blog.cloudflare.com/next-gen-web3-network/ ). In fact they "bet on blockchain" way back in 2017 ( https://blog.cloudflare.com/betting-on-blockchain/ ) but it's tellin…
Im neutral on crypto, I see it like AI, its just waiting on some breakthrough that pulls everyone. My suspicion is someone needs to make it stupid easy to get into crypto.
Crypto is actually pretty useful and common in some marginalised communities where payment processors usually refuse to service (e.g. some sex stuff).
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#246Earlier quoted context omitted.
>Because it's a computer that automatically does it. That's the entire problem here. Humans are not in the loop, except collecting the paychecks. I don't see how "Humans are not in the loop" is a relevant factor for whether something is a "criminal enterprise" or not. Humans are often not in the loop in approving loans/credit cards either. That doesn't make equifax a "criminal enterprise" for blocking you from gettin…
> At least looking at the response headers, digikey.com is served by akamai, not cloudflare I edited them out because they were only one of many problem sites. > Maybe there's something extra sus about your connection/browser, but I find it hard to believe that you have to resort to getting a separate computer and making a 10 minute trek to visit a site Maybe half a decade ago someone had malware from my IP. Maybe my…
>I edited them out because they were only one of many problem sites.
But the fact that other security providers flagged your IP/browser should be enough to conclude that cloudflare isn't engaged in some sort of "protection racket" to extract money from you?
>The vendor in question (this one was not digikey) very explicitly wanted me as a customer.
Most e-commerce vendors also want customers as well, the problem they can't tell an anonymous visitor a legitimate customer or not, so they employ security services like cloudflare to do that for them.
>Them charging to correct their mistake is.
It's unclear whether the cloudflare product actually constitutes "Them charging to correct their mistake". For one, it's unclear whether you're blocked by cloudflare or the site owner, who can also set rules for blocking/challenging users. Moreover, it's unknown whether the website owner would opt into this marketplace. Presumably they're blocking bots for fraud/anti-competition reasons. If that's the case I doubt they're going to put their sites up for scraping to make a few bucks. Finally, businesses are under no obligation to give you free appeals, so the inability for you to freely appeal doesn't constitute a "protection racket".
>vanilla android on t-mobile periodically gets the infinite loop
>I have seen it on linux windows and android.
you must have a really dodgy IP block then.
>I don't understand this. They have a normal ISP in a business district?
Its probably generating two signals associated with fraud:
1. high latency means than a proxy is being used. This is suspicious because customers typically don't VPN themselves halfway across the world, but cybercriminals trying to cover their tracks by using residential proxies do
2. "business" ISPs might get binned as "hosting" providers, which is also suspicious for similar reasons (eg. could be someone using a VPS as a proxy).
Sure, the unlucky few who accidentally does some online shopping when connected to their work VPN might get falsely flagged, but they probably figure it's a rare enough case that it's worth the loss compared to the overwhelming amount of fraudsters that fit the same pattern.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#247Earlier quoted context omitted.
> Those "keep clicking until we stop fading in more results" challenges mean they're fairly confident you're a bot Those ones are the fucking worst. I've noticed that if I try to succeed in these captchas too quickly, it'll just say "Sorry, try again" even when every click was correct, so instead, I've started going in slow motion and faking "misclicking" which makes it much more likely to accept me as human. I canno…
I always spoil as many of these as possible. Sometimes it takes me a while to prove that I'm human, but I'm dead-set on convincing it that I'm a stupid human. Of course, I fantasize that some day a robo-car will crash because I taught it that there's really no difference between a motorcycle and a flight of stairs.
https://qntm.org/frame
Excellent short story that’s, somewhat related at least.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#248Earlier quoted context omitted.
Those "keep clicking until we stop fading in more results" challenges mean they're fairly confident you're a bot and this is the highest difficulty level to prove your lack of guilt. I get these only when using a browser that isn't already full of advertising cookies (edit: which, to be clear, I hope is still considered an acceptable state to have your browser in)
FWIW, it can't be cookies alone that gives you an inordinate number of bot challenges. I use private tabs on Firefox (for Linux and Android) for most of my browsing, and I rarely get any challenges regardless of what I do. The only issues tend to be when I make repeated searches for things with "quotes" and whatnot on Google or on Stack Exchange sites. But for the most part, those challenges aren't particularly drawn…
>or a vpn
My wife does not get these captchas yet I do, on the same network. I have more privacy enhancing software on my devices. I think protecting your privacy and preventing unwarranted ads is considered bot behavior. This should absolutely be villainized and banned from practice
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#249I was recently speaking with people from OpenFoodFacts and OpenStreetMap, and I guess Wikipedia as the same issue. They are under constantly DDoS by bots which are scraping everything, even if the full dataset can be downloaded for free with a single HTTP request. They said this useless traffic was a huge cost for them. This is not about copyright, just about bots being stupid and people behind them not caring at all…
To be fair, some 20 years ago when I wanted to do something with Wikipedia data, I scraped them too, after having tried quite a bit to use the dumps. - dump availability was shaky at best back then (could see months go by without successful dumps) - you had to fiddle with it to actually process the dumps - you'd get the full wikipedia content, but you didn't have the exact wikipedia mediawiki setup, so a bunch of thi…
Can relate. I've used their dumps, and one task was to generate a paragraph summary. The dumps themselves use wiki markup which obviously adds an entirely new level of complexity. There are dumps of "summaries" but they're fairly broken, seemingly due to an ever evolving wiki markup syntax. I believe there are other ways to parse them though, which involves downloading a bunch of other people's code.
So if someone were to scrape the front end for the first paragraph element or whatever, it may make their life easier.
Re: Cloudflare's new marketplace lets websites charge AI bots for scraping
#250Common Crawl is shown in their screen shot of "Providers" along side OpenAI and Antropic. The challenge is that Common Crawl is used for a lot of things that are not AI training. For example, it's a major source of content for the Wayback machine. In fact, that's the entire point of the Common Crawl project. Instead of dozens of companies writing and running their (poorly) designed crawlers and hitting everyone's sit…
> gatekeep access to those who pay and those who don't, and that applied whether they are bots or people. I'm already constantly being classified as bot. Just today: To check if something is included in a subscription that we already pay for, I opened some product page on the Microsoft website this morning. Full-page error: "We are currently experiencing high demand. Please try again later." It's static content but i…
Microsoft might just be a functional bug, that sounds consistent with the rest of their offerings.