I know people have opinions about Cloudflare but why not use it here, at least as a stop gap? Stopping bot traffic is one thing it does very well.
What happened to TheNumbers.com
31–40 of 218 posts
Re: What happened to TheNumbers.com
#32Earlier quoted context omitted.
But it also destroys the business model behind the site. Sure, you could technically redesign to handle the bot traffic, but if the bot traffic is just taking the data and reducing any need for humans to visit the site, why is he putting in the effort to maintain the site?
Well, why is he? If it's for ad revenue, this problem is surely global to the web. If it's because he likes to do it, it shouldn't matter if one AI or a trillion scrape the site. I feel like we badly need a rethink of the web architecture after DoubleClick anyway. Maybe the name of the game should be to cut down a site's assets very hard and use static hosting for them. This interferes with crummy sites that show a m…
Maybe more an inherent problem with these chatbots.
LLMs are great, but they aren't producing new information. You still need people for that. But if you cut down any incentive for the people to do that, the LLMs will starve.
Re: What happened to TheNumbers.com
#33"One Reddit theory even suggested it was a deliberate rug pull designed to cripple the free site to push people towards paid products." I know this isn't really the point of the article, but I've been thinking about this a lot. I wonder if we're going to see more resources go this way. I used to publish little doo-dads as opensource software. Not because it was something that was legitimately ground breaking or anyth…
This attitude makes zero sense to me. You benefit from the "training data" just like everyone else does. If you don't, that's a problem with you, not a problem with AI models. AI solves exactly the meta-problem you describe: "I had a problem and needed to write a one-off doo-dad utility program to solve it." Now you can do something with your time besides writing pointless one-off doo-dads. As for monetizing the trai…
[dead]
Re: What happened to TheNumbers.com
#34"One Reddit theory even suggested it was a deliberate rug pull designed to cripple the free site to push people towards paid products." I know this isn't really the point of the article, but I've been thinking about this a lot. I wonder if we're going to see more resources go this way. I used to publish little doo-dads as opensource software. Not because it was something that was legitimately ground breaking or anyth…
This attitude makes zero sense to me. You benefit from the "training data" just like everyone else does. If you don't, that's a problem with you, not a problem with AI models. AI solves exactly the meta-problem you describe: "I had a problem and needed to write a one-off doo-dad utility program to solve it." Now you can do something with your time besides writing pointless one-off doo-dads. As for monetizing the trai…
| Now you can do something with your time besides writing pointless one-off doo-dads.
...thus eliminating all the tedious chatting, relaxing and making friends that people were previously forced to do while waiting for elevators.
Re: What happened to TheNumbers.com
#35What does a cyber attack have to do with AI scraping?
Re: What happened to TheNumbers.com
#36"One Reddit theory even suggested it was a deliberate rug pull designed to cripple the free site to push people towards paid products." I know this isn't really the point of the article, but I've been thinking about this a lot. I wonder if we're going to see more resources go this way. I used to publish little doo-dads as opensource software. Not because it was something that was legitimately ground breaking or anyth…
I open source as much software as I can because I want the models to train on it and get better at it!
Re: What happened to TheNumbers.com
#37I know people have opinions about Cloudflare but why not use it here, at least as a stop gap? Stopping bot traffic is one thing it does very well.
Cost?
Re: What happened to TheNumbers.com
#38Ironic that polymarkets were being advertised as helping society make better predictions.
I mean wisdom of crowds, super-forecasters, calibration and pre-registration are useful tools that can result in better predictions, turning it into online gambling is where it went sideways.
Re: What happened to TheNumbers.com
#39Earlier quoted context omitted.
It would not make any difference.
Indeed, the article mentions Wikipedia experiencing similar scraping pains, even though they already DO have bulk data available.
Who are running these bots? I presume developers at all of the frontier labs know (or at least would know to look for) Wikipedia has bulk APIs for automated access. Unnecessary scraping increases their workload/costs too, so why in 2026 is this still a problem?
Re: What happened to TheNumbers.com
#40Earlier quoted context omitted.
This attitude makes zero sense to me. You benefit from the "training data" just like everyone else does. If you don't, that's a problem with you, not a problem with AI models. AI solves exactly the meta-problem you describe: "I had a problem and needed to write a one-off doo-dad utility program to solve it." Now you can do something with your time besides writing pointless one-off doo-dads. As for monetizing the trai…
| Now you can do something with your time besides writing pointless one-off doo-dads. ...thus eliminating all the tedious chatting, relaxing and making friends that people were previously forced to do while waiting for elevators.
...thus eliminating all the tedious chatting, relaxing and making friends that people were previously forced to do while waiting for elevators
That also makes no sense, but I don't know what else I should have expected.