Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codex scraped the ICM website and discovered 2026 Fields Medal winner list

phemex.com

Post reply on HN
Print view View original

11–20 of 119 posts

Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list

#11

Post by foobarbecue »

resonious wrote: post #10
edoceo wrote: post #8

I've been working on a site. It's new, domain is only a few weeks old. It's got SSL, so all the bots know it exists. It's never had any sub-pages exposed, just the placeholder lander, no links. Somehow in Google search one of the unguessable pages is indexed. We have used Claude and Gemini to assist with some design aspects. I'm thinking some aggressive data ingestion/indexing is happening by all the bots in the ques…

I've also seen Google indexing pages with random values in the path that don't get linked to statically (server asks for the URL then redirects to it immediately). I'm pretty sure they index straight out of the Chrome address bar.

Holy crap I hope that's not true. I've also had unguessable pages indexed, though, and don't have an explanation.

Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list

#12

Post by nicce »

foobarbecue wrote: post #11
resonious wrote: post #10

Earlier quoted context omitted.

I've also seen Google indexing pages with random values in the path that don't get linked to statically (server asks for the URL then redirects to it immediately). I'm pretty sure they index straight out of the Chrome address bar.

Holy crap I hope that's not true. I've also had unguessable pages indexed, though, and don't have an explanation.

Something worth inspecting further. We know that Chrome stores and sends the browsing history but this is an interesting vector.

Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list

#15

Post by grommz »

bananaflag wrote: post #4
rurban wrote: post #3

It's Wang Hong, my god. Cannot they still don't write proper Chinese names?

Wikipedia says Hong Wang while acknowledging that the native form is Wang Hong and that they are using the Western name order.

Nobody says Jinping Xi or Zedong Mao.

Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list

#16

Post by st_goliath »

resonious wrote: post #10
edoceo wrote: post #8

I've been working on a site. It's new, domain is only a few weeks old. It's got SSL, so all the bots know it exists. It's never had any sub-pages exposed, just the placeholder lander, no links. Somehow in Google search one of the unguessable pages is indexed. We have used Claude and Gemini to assist with some design aspects. I'm thinking some aggressive data ingestion/indexing is happening by all the bots in the ques…

I've also seen Google indexing pages with random values in the path that don't get linked to statically (server asks for the URL then redirects to it immediately). I'm pretty sure they index straight out of the Chrome address bar.

Yep. I remember a similar story as GP described from a friend back in 2008. The site he was working on that wasn't linked to yet was suddenly indexed after he checked out what it looked like in the fancy new "Chrome" browser that Google had just released, causing some moderate panic on his end.

Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list

#17

Post by dreambigwrkhard »

edoceo wrote: post #8

I've been working on a site. It's new, domain is only a few weeks old. It's got SSL, so all the bots know it exists. It's never had any sub-pages exposed, just the placeholder lander, no links. Somehow in Google search one of the unguessable pages is indexed. We have used Claude and Gemini to assist with some design aspects. I'm thinking some aggressive data ingestion/indexing is happening by all the bots in the ques…

Depending on the CMS, if it's wordpress (15% chance, ha) there is a sitemap function built-in out of the box. The bots don't need to guess.

Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list

#18

Post by st_goliath »

ashu1461 wrote: post #2

Someone used Codex to scrape the ICM website schedule and discovered that the winners list was simply hidden in the front-end code with a "hidden" tag This is on the devs and feels like a very basic leak which could have exploited in the non LLM world as well.

Well, the angle is kind of important here. The company gets their name in the news, they have a reasonable explanation why they were scraping around, and we end up with a story about innovative tech company whiz-kids who made a funny discovery, while it was the webdevs on the other side that goofed up.

Imagine a private individual just scraped the website (or simply clicked 'view source') for no reason in particular and then told people about it... They'd be labeled an uber-haxxor, face a civil lawsuit asking for ridiculous damages while being threatened with a prison sentence over CFAA violations. Hell, that might even drive some people to suicide.

Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list

#20

Post by cynerx »

edoceo wrote: post #8

I've been working on a site. It's new, domain is only a few weeks old. It's got SSL, so all the bots know it exists. It's never had any sub-pages exposed, just the placeholder lander, no links. Somehow in Google search one of the unguessable pages is indexed. We have used Claude and Gemini to assist with some design aspects. I'm thinking some aggressive data ingestion/indexing is happening by all the bots in the ques…

Are you using Cloudflare by any chance? I think the Crawler Hints setting [1] exposed some of my "secret" pages in the past.

[1] https://developers.cloudflare.com/cache/advanced-configurati...

Post reply on HN