Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare Introduces Default Blocking of A.I. Data Scrapers

nytimes.com

Post reply on HN
Print view View original

321–330 of 342 posts

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#321

Post by Dylan16807 »

reaperducer wrote: post #320
Dylan16807 wrote: post #318

Earlier quoted context omitted.

Where did you get that date? https://serverfault.com/questions/171985/how-can-i-encourage... Here's a 2010 discussion about Google's explicit support, and I'm sure I could find earlier. The thing google did in 2019 was submit it as a standard, nothing to do with adoption or starting to recommend. In that very post they said "For 25 years, the Robots Exclusion Protocol (REP) has been one of the most basic and critical…

Where did you get that date? On July 1, 2019, Google announced the proposal of the Robots Exclusion Protocol as an official standard under Internet Engineering Task Force.[8] https://en.wikipedia.org/wiki/Robots.txt

That is not when they started recommending it. It would be nice if you acknowledged the rest of my comment, I even quoted from the [8] reference.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#322

Post by MichaelZuo »

Dylan16807 wrote: post #319
MichaelZuo wrote: post #316

Earlier quoted context omitted.

None of this even makes sense. Why does your opinion on how a comment should be interpreted, matter more than anyone else’s opinion in the first place?

I didn't say that. Huh? I'm inviting you to tell me how to interpret it. In fact I'm nearly begging you to explain your original comment more. I'm not telling anyone how to interpret it. I have criticisms for what was said, but that comes after (attempted) interpretation and builds on top of it. I'm not telling anyone how to interpret any post I didn't make. Edit: In particular, my previous comment has "I assumed" to…

This is even closer to gibberish… what are you even trying to say?

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#323

Post by Dylan16807 »

MichaelZuo wrote: post #322
Dylan16807 wrote: post #319

Earlier quoted context omitted.

I didn't say that. Huh? I'm inviting you to tell me how to interpret it. In fact I'm nearly begging you to explain your original comment more. I'm not telling anyone how to interpret it. I have criticisms for what was said, but that comes after (attempted) interpretation and builds on top of it. I'm not telling anyone how to interpret any post I didn't make. Edit: In particular, my previous comment has "I assumed" to…

This is even closer to gibberish… what are you even trying to say?

You don't understand a word I'm saying, and you have missed/declined every single time I asked you to explain the first comment I responded to.

Let's just mutually give up on this conversation.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#324

Post by giancarlostoro »

fkyoureadthedoc wrote: post #148
giancarlostoro wrote: post #141

Earlier quoted context omitted.

Ah yes, its only because of ONE singular reason they started charging for API usage. Are you okay? I'm listing one reason out of many as to why reddit started charging for API usage. After all, reddit is a for profit website.

> There's *A* reason this u chief?

Does "a" not mean, one out of many in this context? I know English is not my first language, but I've always taken "a reason" to mean one of many.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#325

Post by const_cast »

lxgr wrote: post #201
sealeck wrote: post #197

Earlier quoted context omitted.

But Cloudflare have removed CAPTCHAs

Not sure if you're joking, but if you're not: Congratulations on using a very "normal/safe" OS/browser/IP. I get captchas daily, without using any VPN and on several different IPs (work, home, mobile). The only crime I can think of is that I'm using Firefox instead of Chrome.

Really? Because I'm on Debian, with Firefox, with a VPN active 24/7 and I almost never get Captchas. I do get those "checking your browser" pages often but they just stick around for maybe half a second then redirect.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#326

Post by godelski »

jefftk wrote: post #300
godelski wrote: post #289

Earlier quoted context omitted.

> The way a license works is Let's actually look at the MIT license, a very permissive license > Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to ***use***, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of t…

> Forget the courts for a second, just ask yourself what is the right thing to do Forgetting the courts, whether reading the source code and learning from it is intended to count as "use" is not clear to me, and I would have guessed no. Using a tool and examining a tool are pretty different.

Context matters, right?

Human reading code? Ambiguous. But I think you're using it. Running code? Not ambiguous.

Machine processing code? I don't think that's ambiguous. It's using the code. A person is using the code to make their machine better.

This really isn't that hard.

Let's think about it this way. How do you use a book?

I think you need to be careful that you're not justifying the answer you want and instead are looking for what the right answer is. I'm saying this because you quoted me saying "what is right" and you just didn't address it. To quote Feynman (

  > The first principle is that you must not fool yourself, and you are the easiest person to fool.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#327

Post by ranger_danger »

lurking_swe wrote: post #256
slenk wrote: post #241

Earlier quoted context omitted.

Dude, stop putting words in my mouth. I never said they weren't bad. Some nicer people here tried the educative approach and it worked much better. I learned about Bunny. And I keep forgetting I have a few in deSec but that has a limit. I do not understand the hostility

> I do not understand the hostility Unfortunately I don’t think they were participating in the conversation in good faith. People can have an extreme view on _anything_…even internet / tech. They buy into a dream of 100% open source, or “open internet”, or 100% decentralized, whatever. When this happens they may be convinced that “others” are crazy for not sharing their utopian vision. And once this point is reached,…

One of my favorite quotes: "As a rule, strong feelings about issues do not emerge from deep understanding." -Sloman and Fernbach

Learning how to spot this, and ignore such-minded people who argue in bad faith, has made me a lot happier and more chill in general.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#328

Post by MichaelZuo »

Dylan16807 wrote: post #323
MichaelZuo wrote: post #322

Earlier quoted context omitted.

This is even closer to gibberish… what are you even trying to say?

You don't understand a word I'm saying, and you have missed/declined every single time I asked you to explain the first comment I responded to. Let's just mutually give up on this conversation.

Okay then.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#329

Post by 1oooqooq »

mattl wrote: post #247
1oooqooq wrote: post #221

Earlier quoted context omitted.

it's called not having a vibecoded app that falls to pieces on public endpoints even before ngix ratelimit can kick in

Nobody is talking about a vibe coded app. I want to block AI scrapers entirely.

point is, why do you care if your site can handle the traffic?

there's no (malicious) bot detection that won't impact a portion of real users. accept that fact and just let it be.

poisoning data in ways that's obvious to the false positive user is a much better option.

Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers

#330

Post by fsflover »

1oooqooq wrote: post #219
fsflover wrote: post #133

Earlier quoted context omitted.

They put themselves as a middle man for almost the whole Internet, collect huge usage data about everyone and block anybody who doesn't use mainstream tools: https://news.ycombinator.com/item?id=42953508 https://news.ycombinator.com/item?id=13718752 https://news.ycombinator.com/item?id=23897705 https://news.ycombinator.com/item?id=41864632 https://news.ycombinator.com/item?id=42577076

https://infosec.exchange/@k3ym0/114762301792775770

https://en.wikipedia.org/wiki/Cloudflare#Controversies
Post reply on HN