361–370 of 668 posts
Earlier quoted context omitted.
I bet dollars to doughnuts that 95% of the traffic is from Claude and ChatGPT desktop / mobile and not literal content scraping for training.
That wouldn't explain the 1000x increase in traffic for extremely obscure content, or seeing it download every single page on a classic web forum.
I just don't understand why bot owners can't just run a complete windows 11 VM running Google Chrome complete with graphics acceleration. You can probably run 50 of those simultaneously if you use memory page deduplication, and with a decent CPU+GPU you ought to be able to render 50 pages a second. That's 1 cent per thousand page loads on AWS. Damn cheap.
Earlier quoted context omitted.
> Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things. I bet people being fucking DDOSed by AI bots disagree Also the fucking ignorance assuming it's "static content" and not something needing code running
> I bet people being fucking DDOSed by AI bots disagree Are you sure it's a DDoS and not just a DoS?
We implemented an anti-bot challenge and it helped for a while. Then our server collapsed again recently. The perf command showed that the actual TLS handshakes inside nginx were using over 50% of our server's CPU, starving other stuff on the machine.
It's a DDoS.
Earlier quoted context omitted.
It's still possible to let users already type from the beginning, just delay sending the characters until checks are complete. Hold them in memory until then.
Instagram was uploading the images while the user were adding post details, back in 2012! https://news.ycombinator.com/item?id=3913919 No one seem to use or care about their own product anymore. Only uses dashboard and metrics, which does not explain the full situation.
Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform. A big reason we invest in this is because we want to keep free and logged-out access available for more users. My team’s goal is to help make sure the limited GPU resources are going to real users. We also keep a very close…
Earlier quoted context omitted.
It's interesting to me that OpenAI considers scraping to be a form of abuse.
Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things. The former relies on fairly controversial ideas about copyright and fair use to qualify as abuse, whereas the latter is direct financial damage – by your own direct competitors no less. It's fun to poke at a seeming hypocrisy of the big bad, but the similarity…
Earlier quoted context omitted.
It's interesting to me that OpenAI considers scraping to be a form of abuse.
Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things. The former relies on fairly controversial ideas about copyright and fair use to qualify as abuse, whereas the latter is direct financial damage – by your own direct competitors no less. It's fun to poke at a seeming hypocrisy of the big bad, but the similarity…
Earlier quoted context omitted.
Because OpenAI built their entire business around shamelessly scraping anything that had bits on it.
Maybe. But scraping isn't abuse. Seems a bit different?
> These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
That implies that OpenAI (or at least this employee) considers scraping abuse.
Earlier quoted context omitted.
Because OpenAI built their entire business around shamelessly scraping anything that had bits on it.
Maybe. But scraping isn't abuse. Seems a bit different?