What happened to TheNumbers.com • Hacker News Board

Re: What happened to TheNumbers.com

#21

Post by hyperhello » Thu, Jul 23, 2026, 6:19 PM UTC

jaredwiener wrote: post #17

tengada1 wrote: post #7

Earlier quoted context omitted.

Yeah this seems like one of the easiest access models to create an excellent security model for -- basically just static publishing. Sounds like they were in need of a rewrite anyway! It probably seems daunting but to be honest this feels like a weekend's work at this point with LLM assistance. Not to be glib!

But it also destroys the business model behind the site. Sure, you could technically redesign to handle the bot traffic, but if the bot traffic is just taking the data and reducing any need for humans to visit the site, why is he putting in the effort to maintain the site?

Well, why is he? If it's for ad revenue, this problem is surely global to the web. If it's because he likes to do it, it shouldn't matter if one AI or a trillion scrape the site. I feel like we badly need a rethink of the web architecture after DoubleClick anyway. Maybe the name of the game should be to cut down a site's assets very hard and use static hosting for them. This interferes with crummy sites that show a mess of inlined ads every refresh, but now that there are a billion 'poor users' this is no longer feasible.

Re: What happened to TheNumbers.com

#22

Post by [deleted] » Thu, Jul 23, 2026, 6:22 PM UTC

nater5000 wrote: post #11

This is an odd example to present this argument through. I'm not defending AI bots overwhelming websites or hackers motivated by Polymarket, etc., but I don't really believe a 30 year old website with "approximately 160,000 source files serving around 2 million pages" is a good litmus test for the state of the online world. What's worse is the absurdity that this basic site offering niche data would be targeted becau…

[deleted]

Re: What happened to TheNumbers.com

#23

Post by hebleb » Thu, Jul 23, 2026, 6:26 PM UTC

Good read, I was so curious how this happened a few months ago

Re: What happened to TheNumbers.com

#24

Post by tehjoker » Thu, Jul 23, 2026, 6:30 PM UTC

The real story here is that prediction markets were banned for a reason and loosening the rules is causing chaos just as was expected. AI plays little role in this story, hacking by humans would also be motivated by financial returns, unless the element is that AI hacking is cheaper and the returns are not so big.

Re: What happened to TheNumbers.com

#25

Post by primitivesuave » Thu, Jul 23, 2026, 6:32 PM UTC

A couple years ago, I was running a website which allowed the public to view all the US government handouts to small businesses during the COVID-19 pandemic. It also tracked all the fraudulent loans being prosecuted by the DOJ, and allowed anyone to run structured queries over the public dataset. There was a "donate" button which took in ~$2k in donations over the lifetime of the site, and you could download the entire underlying dataset (around 10 GB uncompressed) for free directly on the site.

Despite the "download all data" link being prominently placed on the front page, the AI scrapers decided it would be more efficient to download terabytes upon terabytes of raw HTML by paginating through every possible facet on the search endpoint. Even with CloudFront caching results and a fairly efficient backend setup, the monthly bill ended up with around $1k just going toward network ingress/egress, so I shut down the site the following month.

Re: What happened to TheNumbers.com

#26

Post by RobotToaster » Thu, Jul 23, 2026, 6:33 PM UTC

ethagnawl wrote: post #4

At the risk of oversimplifying things from a distance, this site -- especially the free, public-facing part of it -- seems like it would be an ideal candidate for a rewrite using static site generator/framework. That, coupled with a bot-aware CDN should keep them online at a reasonable cost for many years to come. Otherwise, I'm very curious to know more about their old and new architecture and what sorts of mitigati…

Or just use a cache.

Re: What happened to TheNumbers.com

#27

Post by podgietaru » Thu, Jul 23, 2026, 6:33 PM UTC

"One Reddit theory even suggested it was a deliberate rug pull designed to cripple the free site to push people towards paid products."

I know this isn't really the point of the article, but I've been thinking about this a lot. I wonder if we're going to see more resources go this way.

I used to publish little doo-dads as opensource software. Not because it was something that was legitimately ground breaking or anything (it absolutely wasn't) but because I had a problem, and I thought "heh wouldn't it be cool if someone else had a similar problem and could use my resource for it."

But now I'm really reluctant to give more stuff to the free web. Because the fact that it gets scraped and added to a pile of training data to later be monetized really rubs me the wrong way.

And I can see why maintainers of sites like this, or other free but incredibly useful resources might start to get irate at that.

Re: What happened to TheNumbers.com

#28

Post by jambalaya8 » Thu, Jul 23, 2026, 6:34 PM UTC

dumberquestions wrote: post #3

Ironic that polymarkets were being advertised as helping society make better predictions.

John Brunner and Alvin Toffler both made stark warnings wrapped in futuristic giddiness about things like it. People like Fuller no doubt thought polling on a large scale was terrific. There were old usenet groups and BBS subs (minus the money aspect) experimenting with the model. I do not believe they ever are or were good in a largescale model (money or not).

Re: What happened to TheNumbers.com

#29

Post by CamperBob2 » Thu, Jul 23, 2026, 6:36 PM UTC

podgietaru wrote: post #27

"One Reddit theory even suggested it was a deliberate rug pull designed to cripple the free site to push people towards paid products." I know this isn't really the point of the article, but I've been thinking about this a lot. I wonder if we're going to see more resources go this way. I used to publish little doo-dads as opensource software. Not because it was something that was legitimately ground breaking or anyth…

This attitude makes zero sense to me. You benefit from the "training data" just like everyone else does. If you don't, that's a problem with you, not a problem with AI models.

AI solves exactly the meta-problem you describe: "I had a problem and needed to write a one-off doo-dad utility program to solve it." Now you can do something with your time besides writing pointless one-off doo-dads.

As for monetizing the training data, (a) it cost hundreds of millions of dollars to generate the weights, so why begrudge the companies that made the investment and did the research necessary to make it happen?; and (b) rest assured, whatever your doo-dad does, an open-weight model like GLM 5.2 can generate it for free using your own hardware.

So you don't have to pay anyone in that case. Well, except nVidia, I guess. Point granted there.

Re: What happened to TheNumbers.com

#30

Post by jambalaya8 » Thu, Jul 23, 2026, 6:36 PM UTC

BraveOPotato wrote: post #12

jambalaya8 wrote: post #9

I always liked this site, but reading this and seeing the anger about expecting the site maintainer to do things for you is repulsive. Frankly, if he wanted to pull his site down with no notice that is perfectly within his right. It was/is his site. He doesn't owe anyone a .tar.gz either. His work.

I agree. I've seen it happen before on a project I use. I decided to take a look at the repo for one of the plugins, and I saw a heinous issue that basically was TELLING (not even asking) the maintainer to fix it. Deplorable behavior indeed

Like dominoes, as soon as it is accepted in a few places, people think it is acceptable to push to 'share'. It's almost terroristic sometimes, the pressure some maintainers are under.