Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT won't let you type until Cloudflare reads your React state

buchodi.com

Post reply on HN
Print view View original

411–420 of 668 posts

Re: ChatGPT won't let you type until Cloudflare reads your React state

#411

Post by miki123211 »

stingraycharles wrote: post #280
not2b wrote: post #248

Earlier quoted context omitted.

I understand why OpenAI is trying to reduce its costs, but it simply isn't true that AI crawlers aren't creating very significant load, especially those crawlers that ignore robots.txt and hide their identities. This is direct financial damage and it's particularly hard on nonprofit sites that have been around a long time.

These are ChatGPT and Claude Desktop crawlers we’re talking about? Or what is it exactly? Are these really creating significant load while not honoring robots.txt? Genuinely interested.

They seem to mostly be third-party upstarts with too much money to burn, willing to do what it takes to get data, probably in hopes of later selling it to big labs. Maaaybe Chinese AI labs too, I wouldn't put it past them.

OpenAI et al seem to mostly be well-behaved.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#412

Post by lelanthran »

eloisius wrote: post #356
PunchyHamster wrote: post #256

Earlier quoted context omitted.

> Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things. I bet people being fucking DDOSed by AI bots disagree Also the fucking ignorance assuming it's "static content" and not something needing code running

Also wild that from the tech bro perspective, the cost of journalism is just how much data transfer costs for the finished article. Authors spend their blood, sweat and tears writing and then OpenAI comes to Hoover it up without a care in the world about license, copyright or what constitutes fair use. But don’t you dare scrape their slop.

> Also wild that from the tech bro perspective, the cost of journalism is just how much data transfer costs for the finished article.

Exactly. I think the unfairness can be mitigated if models trained on public information, or on data generated by a model trained on public information, or has any of those two in its ancestry, must be made public.

Then we don't have to hit (for example) Anthropic, we can download and use the models as we see fit without Anthropic whining that the users are using too much capacity.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#413

Post by dmos62 »

Chance-Device wrote: post #9

Perhaps the author should have made it clearer why we should care about any of this. OpenAI want you to use their real react app. That’s… ok? I skimmed the article looking for the punchline and there doesn’t seem to be one.

For me the interesting parts of the article is how author got to the decompiled checks and what the checks are. Anti-bot is an interesting space.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#414

Post by miki123211 »

sandeepkd wrote: post #277
nikitaga wrote: post #217

Earlier quoted context omitted.

Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things. The former relies on fairly controversial ideas about copyright and fair use to qualify as abuse, whereas the latter is direct financial damage – by your own direct competitors no less. It's fun to poke at a seeming hypocrisy of the big bad, but the similarity…

Lets not try to qualify the wrongs by picking a metric and evaluating just one side of it. A static website owner could be running with a very small budget and the scraping from bots can bring down their business too. The chances of a static website owner burning through their own life savings are probably higher.

If you're truly running a static site, you can run it for free, no matter how much traffic you're getting.

Github pages is one way, but there are other platforms offering similar services. Static content just isn't that expensive to host.

THe troubles start when you're actually running something dynamic that pretends to be static, like Wordpress or Mediawiki. You can still reduce costs significantly with CDNs / caching, but many don't bother and then complain.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#415

Post by lelanthran »

Aurornis wrote: post #232
Imnimo wrote: post #149

Earlier quoted context omitted.

It's interesting to me that OpenAI considers scraping to be a form of abuse.

I interpreted scraping to mean in the context of this: > we want to keep free and logged-out access available for more users I have no doubt that many people see the free ChatGPT access as a convenient target for browser automation to get their own free ChatGPT pseudo-API.

> I have no doubt that many people see the free ChatGPT access as a convenient target for browser automation to get their own free ChatGPT pseudo-API.

Not that hard - ChatGPT itself wrote me a FF extension that opened a websocket to a localhost port, then ChatGPT wrote the Python program to listen on that websocket port, as well as another port for commands.

Given just a handful of commands implemented in the extension is enough for my bash scripts to open the tab to ChatGPT, target specific elements, like the input, add some text to it, target the relevant chat button, click it, etc.

I've used it on other pages (mostly for test scripts that don't require me to install the whole jungle just to get a banana, as all the current playright type products do). Too afraid to use it on ChatGPT, Gemini, Claude, etc because if they detect that the browser is being drive by bash scripts they can terminate my account.

That's an especially high risk for Gemini - I have other google accounts that I won't want to be disabled.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#416

Post by supriyo-biswas »

iancarroll wrote: post #306
petcat wrote: post #4

> These properties only exist if the ChatGPT React application has fully rendered and hydrated. A headless browser that loads the HTML but doesn't execute the JavaScript bundle won't have them. A bot framework that stubs out browser APIs but doesn't actually run React won't have them. > This is bot detection at the application layer, not the browser layer. I kind of just assumed that all sophisticated bot-detectors a…

It’s pretty interesting to me that Cloudflare is collecting additional client-side data for individual customers. This is not widely done by most anti-bot solutions.

OpenAI is on an enterprise plan and (presumably) gets a customized version of Turnstile.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#417

Post by raincole »

Chance-Device wrote: post #9

Perhaps the author should have made it clearer why we should care about any of this. OpenAI want you to use their real react app. That’s… ok? I skimmed the article looking for the punchline and there doesn’t seem to be one.

Why does every article need a 'punchline'? It's a technical analysis. Do you expect punchlines when you read recipes or source code?

Re: ChatGPT won't let you type until Cloudflare reads your React state

#418

Post by 63stack »

stingraycharles wrote: post #280
not2b wrote: post #248

Earlier quoted context omitted.

I understand why OpenAI is trying to reduce its costs, but it simply isn't true that AI crawlers aren't creating very significant load, especially those crawlers that ignore robots.txt and hide their identities. This is direct financial damage and it's particularly hard on nonprofit sites that have been around a long time.

These are ChatGPT and Claude Desktop crawlers we’re talking about? Or what is it exactly? Are these really creating significant load while not honoring robots.txt? Genuinely interested.

Is this the first time you are reading HN? Every day there are posts from people describing how AI crawlers are hammering their sites, with no end. Filtering user agents doesn't work because they spoof it, filtering IPs doesn't work because they use residential IPs. Robots.txt is a summer child's dream.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#419

Post by prmoustache »

MyNameIsNickT wrote: post #49

Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform. A big reason we invest in this is because we want to keep free and logged-out access available for more users. My team’s goal is to help make sure the limited GPU resources are going to real users. We also keep a very close…

> we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.

Isn't that how you build your service from the very start? How ironic.

Re: ChatGPT won't let you type until Cloudflare reads your React state

#420

Post by lelanthran »

MyNameIsNickT wrote: post #76
mehov wrote: post #59

Earlier quoted context omitted.

> because we want to keep free and logged-out access But don't you run these checks on logged-in users too?

Yep, on logged-in users too. The reason is basically the same: we want scarce compute going to real people, not attackers. Being logged in is one useful signal, but it doesn’t fully prevent automation, account abuse, or other malicious traffic, so we apply protections in both cases.

> The reason is basically the same: we want scarce compute going to real people, not attackers.

You are defining "Bots" and "Scrapers" as a subset of attackers, though.

Is this really fair? The value in your product came from people who wrote for other people, not bots, but your bot scraped them anyway.

There is no way to determine if a request that is coming from my browser is typed in by me or automated with a browser extension. Your only way to win this "war" on "attackers" is by forcing users into using your own application to access your product.

My browser extension (see my previous reply on this story) automates the existing open tab I have to all the different chat AIs (GPT, Claude, Gemini, etc).

I suppose all you can do is rate-limit each user.

Post reply on HN