I've been working on a site. It's new, domain is only a few weeks old. It's got SSL, so all the bots know it exists. It's never had any sub-pages exposed, just the placeholder lander, no links. Somehow in Google search one of the unguessable pages is indexed. We have used Claude and Gemini to assist with some design aspects. I'm thinking some aggressive data ingestion/indexing is happening by all the bots in the ques…
I've also seen Google indexing pages with random values in the path that don't get linked to statically (server asks for the URL then redirects to it immediately). I'm pretty sure they index straight out of the Chrome address bar.
Codex scraped the ICM website and discovered 2026 Fields Medal winner list
101–110 of 119 posts
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#102Earlier quoted context omitted.
URL paths over https wouldn't be transparent to the ISP though, would they?
They would not - GP was probably bringing up something not directly relevant, but still related. (they should have clarified though)
I have visited that page from a signed-in Chrome profile.
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#103Earlier quoted context omitted.
They would not - GP was probably bringing up something not directly relevant, but still related. (they should have clarified though)
What can I add? I know the domain could be found. But how did Google index a page that is example.com/$MD5.html - guessing 128bit numbers is hard. I have visited that page from a signed-in Chrome profile.
The information you mentioned is relevant. Unfortunately, it could be either Google/Chrome, or the LLM service you're using for development is misusing your data.
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#104Earlier quoted context omitted.
> „popularity“ Why are you using weird quotes?
I'll take a wild guess and assume they are of a German or Polish language background. Wait 'til you encounter a French person who accidentally uses guillemets if you want one even «weirder».
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#105Earlier quoted context omitted.
There's a couple avenues besides just stealing what's in your URL bar. If you don't use wildcard certs all of your subdomains can be scraped from the certificate transparency logs. Additionally, any domain+cert using HSTS with preload enabled end up in a big list at Google to speed up the initial connection from browser to site.
CT logs just explain how they found the domain. T doesn't explain how they could have found unlinked content on the domain itself. If I put up secret-example.com/asdf-1234567.html, how does that page get found if there are no public links to it?
Google misusing chrome browser history as a hitlist for indexing sounds wild to me, so I tried to see if there's another way.
It also felt unlikely because there's multiple subdomains of mine that aren't indexed, and wildcards+no preload are the only precautions I've made myself for my private sites.
This might also be an EU vs rest of World thing, or my stuff isn't interesting enough to index(in retrospect the most likely reason I suppose)
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#106> The leak occurred when four Fields Medal laureate lecture fields, marked "HIDDEN," were discovered in the front-end code of the ICM 2026 official schedule. So it was easier than I thought. Bot just scraped public page with hidden fields, not a secret page or to-be-published page from database.
I'm tired of the framing in the media these days. "Mythos will end the world!!" "How?" "By finding a bunch of wide open security holes that have existed for years." Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#107Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#108> The leak occurred when four Fields Medal laureate lecture fields, marked "HIDDEN," were discovered in the front-end code of the ICM 2026 official schedule. So it was easier than I thought. Bot just scraped public page with hidden fields, not a secret page or to-be-published page from database.
I'm tired of the framing in the media these days. "Mythos will end the world!!" "How?" "By finding a bunch of wide open security holes that have existed for years." Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#109> The leak occurred when four Fields Medal laureate lecture fields, marked "HIDDEN," were discovered in the front-end code of the ICM 2026 official schedule. So it was easier than I thought. Bot just scraped public page with hidden fields, not a secret page or to-be-published page from database.
I'm tired of the framing in the media these days. "Mythos will end the world!!" "How?" "By finding a bunch of wide open security holes that have existed for years." Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?
That's the risk of tools like Mythos/Fable/any LLM. While a human's eyes would glaze over what looks like a standard memcpy, an LLM with the right context might instantly realize the payload length was never actually verified.
And since Heartbleed existed for years, despite the full bug existing in pretty much one file, in one of the most important libraries out there, it's right to be afraid of what other obvious bugs exist and are just waiting to be found.
Re: Codex scraped the ICM website and discovered 2026 Fields Medal winner list
#110> The leak occurred when four Fields Medal laureate lecture fields, marked "HIDDEN," were discovered in the front-end code of the ICM 2026 official schedule. So it was easier than I thought. Bot just scraped public page with hidden fields, not a secret page or to-be-published page from database.
I'm tired of the framing in the media these days. "Mythos will end the world!!" "How?" "By finding a bunch of wide open security holes that have existed for years." Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?