Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT won't let you type until Cloudflare reads your React state

buchodi.com

Post reply on HN
Print view View original

301–310 of 668 posts

Re: ChatGPT won't let you type until Cloudflare reads your React state

#301

Post by freeopinion »

MyNameIsNickT wrote: post #49

Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform. A big reason we invest in this is because we want to keep free and logged-out access available for more users. My team’s goal is to help make sure the limited GPU resources are going to real users. We also keep a very close…

Its your business and your call. But my opinion is that I wish you would quit offering free services. I'm pretty concerned about the horrible effect your free services are having on education. Yes, AI can be an incredible tool to enhance education. But the reality is that it is decimating children's will to learn anything.

I don't want to blame AI for all the world's problems. And I don't want to throw the baby out with the bath water. But I think you should think really hard about the value of gates. Smart people can build better gates than cash. But right now, cash might be better than nothing. Clearly you have already thought about how to build gates, but I don't think you have spent enough time thinking about who should be gated and why. You should think about gates that have more purpose than just maximizing your profit.

"We want to hook as many people as possible without letting in our competitors" is a pretty crummy thought to use as a public justification.

(Edited for typos.)

Re: ChatGPT won't let you type until Cloudflare reads your React state

#302

Post by crote »

cruffle_duffle wrote: post #286
stingraycharles wrote: post #280

Earlier quoted context omitted.

These are ChatGPT and Claude Desktop crawlers we’re talking about? Or what is it exactly? Are these really creating significant load while not honoring robots.txt? Genuinely interested.

I bet dollars to doughnuts that 95% of the traffic is from Claude and ChatGPT desktop / mobile and not literal content scraping for training.

That wouldn't explain the 1000x increase in traffic for extremely obscure content, or seeing it download every single page on a classic web forum.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#303

Post by the_sleaze_ »

nikitaga wrote: post #217
Imnimo wrote: post #149

Earlier quoted context omitted.

It's interesting to me that OpenAI considers scraping to be a form of abuse.

Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things. The former relies on fairly controversial ideas about copyright and fair use to qualify as abuse, whereas the latter is direct financial damage – by your own direct competitors no less. It's fun to poke at a seeming hypocrisy of the big bad, but the similarity…

60% of our traffic is bot, on average. Sometimes almost 100%.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#305

Post by CableNinja »

mystraline wrote: post #171
vkou wrote: post #101

Earlier quoted context omitted.

> Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform. How can first-party products protect themselves from abuse by OpenAI's bots and scraping?

This is a completely in-scope question. How do we defend against your scraping, OpenAI? I dont want any of my content scraped or seen by you all. Frankly, fuck you all for thinking my content is owned by you.

I use nginx conditionals and useragent checking, then respond with 418 or 410.

Probably too late now but my list needs updating

Re: ChatGPT won't let you type until Cloudflare reads your React state

#306

Post by iancarroll »

petcat wrote: post #4

> These properties only exist if the ChatGPT React application has fully rendered and hydrated. A headless browser that loads the HTML but doesn't execute the JavaScript bundle won't have them. A bot framework that stubs out browser APIs but doesn't actually run React won't have them. > This is bot detection at the application layer, not the browser layer. I kind of just assumed that all sophisticated bot-detectors a…

It’s pretty interesting to me that Cloudflare is collecting additional client-side data for individual customers. This is not widely done by most anti-bot solutions.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#307

Post by crote »

heyethan wrote: post #269
Imnimo wrote: post #149

Earlier quoted context omitted.

It's interesting to me that OpenAI considers scraping to be a form of abuse.

[flagged]

Very few websites are truly static. Something like a Wordpress website still does a nontrivial amount of compute and DB calls - especially when you don't hit a cache.

There's also the cost asymmetry to take into account. Running an obscure hobby forum on a $5 / month VPS (or cloud equivalent) is quite doable, having that suddenly balloon to $500 / month is a Really Big Deal. Meanwhile, the LLM company scraping it has hundred of millions of VC funding, they aren't going to notice they are burning a few million because their crappy scraper keeps hammering websites over and over again.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#308

Post by gck1 »

MyNameIsNickT wrote: post #49

Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform. A big reason we invest in this is because we want to keep free and logged-out access available for more users. My team’s goal is to help make sure the limited GPU resources are going to real users. We also keep a very close…

I always wondered why you even have logged out access. I'm glad I can use ChatGPT in incognito when I want a "clean room" response, but surely that's not the primary use case.

Is user base that never logs in really that significant?

Re: ChatGPT won't let you type until Cloudflare reads your React state

#309

Post by zer00eyz »

not2b wrote: post #248
nikitaga wrote: post #217

Earlier quoted context omitted.

Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things. The former relies on fairly controversial ideas about copyright and fair use to qualify as abuse, whereas the latter is direct financial damage – by your own direct competitors no less. It's fun to poke at a seeming hypocrisy of the big bad, but the similarity…

I understand why OpenAI is trying to reduce its costs, but it simply isn't true that AI crawlers aren't creating very significant load, especially those crawlers that ignore robots.txt and hide their identities. This is direct financial damage and it's particularly hard on nonprofit sites that have been around a long time.

> but it simply isn't true that AI crawlers aren't creating very significant load.

And how much of this is users who are tired of walled gardens and enshitfication. We murdered RSS, API's and the "open web" in the name of profit, and lock in.

There is a path where "AI" turns into an ouroboros, tech eating itself, before being scaled down to run on end user devices.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#310

Post by gck1 »

cruffle_duffle wrote: post #287
everdrive wrote: post #69

Earlier quoted context omitted.

It's getting to the point where a user needs at minimum two browsers. One to allow all this horrendous client checking so that crucial services work, and another browser to attempt to prevent tracking users across the web. Nick, I understand the practical realities regarding why you'd need to try to tamp down on some bot traffic, but do you see a world where users are not forced to choose between privacy and function…

There is also the browser I use to get Claude to route around people blocking its webfetch. Both Playwright and chrome-mcp.

Camoufox?
Post reply on HN