Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Watching o3 guess a photo's location is surreal, dystopian and entertaining

simonwillison.net

Post reply on HN
Print view View original

211–220 of 453 posts

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#211

Post by HarHarVeryFunny »

simonw wrote: post #173
HarHarVeryFunny wrote: post #169

Earlier quoted context omitted.

I'm not a computer. I expect a computer to also do better than me at memorizing the phone book, but I'm not impressed by it.

In that case, are you at all surprised that this technology did not exist two years ago?

I'm not sure what you're getting at. What's useful about LLMs, and especially multi-modal ones, is that that you can ask them anything and they'll answer to best of their ability (especially if well prompted). I'm not sure that o3, as a "reasoning" model is adding much value here - since there is not a whole lot of reasoning going on.

This is basically fine-grained image captioning followed by nearest neighbor search, which is certainly something you could have built as soon as decent NN-based image captioning became available, at least 10 years ago. Did anyone do it? I've no idea, although it'd seem surprising if not.

As noted, what's useful about LLMs is that they are a "generic solution", so one doesn't need to create a custom ML-based app to be able to do things like this, but I don't find much of a surprise factor in them doing well at geoguessing since this type of "fuzzy lookup" is exactly what a predict-next-token engine is designed to do.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#212

Post by the8472 »

youniverse wrote: post #201

Does anyone remember that 4chan thing where they geolocated some secret flag location and they used info from planes they saw in the sky or something? I wonder if it could do that now.

https://knowyourmeme.com/memes/he-will-not-divide-us#season-...

screencap collage: https://desuarchive.org/int/thread/72117719/#72133796

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#213

Post by simonw »

HarHarVeryFunny wrote: post #211
simonw wrote: post #173

Earlier quoted context omitted.

In that case, are you at all surprised that this technology did not exist two years ago?

I'm not sure what you're getting at. What's useful about LLMs, and especially multi-modal ones, is that that you can ask them anything and they'll answer to best of their ability (especially if well prompted). I'm not sure that o3, as a "reasoning" model is adding much value here - since there is not a whole lot of reasoning going on. This is basically fine-grained image captioning followed by nearest neighbor search…

How does nearest neighbor search relate to this?

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#214

Post by meowface »

GeoAtreides wrote: post #193
brookst wrote: post #45

I don’t understand the “dystopian” angle. Maybe I’m just old, but I remember the wonder when the Internet made most knowledge available with a few keystrokes. Having deductive reasoning with the same convenience feels wonderful, not dystopian.

That's because you haven't lived in an authoritarian regime. NKVD, STASI, Gestapo, would all have killed for such capabilities. As an east european who grew up and lived in such a regime, I would like to respectfully remind all westerners their care-free and free lives is a privilege the majority of the world doesn't have.

Not to get political, but it deeply irks me to see some American far-leftists glamorize and glorify the Soviet regime and even modern regimes like North Korea's. Especially when certain popular streamers do it. Obviously seeing far-right American internet personalities glorify the Nazi regime is also awful, but the former is often normalized and not considered ostracization-worthy while the latter (rightfully) is.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#215

Post by SamPatt »

tylersmith wrote: post #195
SamPatt wrote: post #178

Earlier quoted context omitted.

As a further test, I dropped the street view marker on a random point in the US, near Wichita, Kansas, here's the image: https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04... I fed it o3, here's the response: https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04... Nailed it. There's no metadata there, and the reasoning it outputs makes perfect sense. I have no doubt it'll be tricky when it can be, b…

This is right by where I grew up and the broadcast tower and turnpike sign were the first two things I noticed too, but the ability to realize it was the East side instead of the West side because the tower platforms are lower is impressive.

Oh hey Tyler, nice to see you on HN :)

Yeah it's an impressive result.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#216

Post by XenophileJKO »

skydhash wrote: post #180
simonw wrote: post #173

Earlier quoted context omitted.

In that case, are you at all surprised that this technology did not exist two years ago?

Did it not, or no one was interested enough to build one? I’m pretty certain there’s a database of portraits somewhere where they search id details from photograph. Automatic tagging exists for photo software. I don’t see why that can be extrapolated to landmarks with enough data.

I think you are underestimating the importance of a "world model" in the process. It is the modeling of how all these details are related to each other that is critical here.

The LLM will have an edge by being able to draw on higher level abstract concepts.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#217

Post by Misdicorl »

simonw wrote: post #121
qarl wrote: post #104

> I’m confident it didn’t cheat and look at the EXIF data on the photograph, because if it had cheated it wouldn’t have guessed Cambria first. It also, at one point, said it couldn't see any image data at all. You absolutely cannot trust what it says. You need to re-run with the EXIF data removed.

I ran several more experiments with EXIF data removed. Honestly though, I don't feel like I need to be 100% robust in this. My key message wasn't "this tool is flawless", it was "it's really weird and entertaining to watch it do this, and it appears to be quite good at it". I think what I've published so far entirely supports that message.

Would be really interesting to see what it does with clearly wrong EXIF data

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#218

Post by gruez »

robrenaud wrote: post #112
tokai wrote: post #94

Earlier quoted context omitted.

Isn't all of streetview in the training set?

O3 is OpenAI. Street view is Google. I really doubt OpenAI is scraping enormous amounts of random street view images to train their model.

Why not? They allegedly trained on enough books and newspapers that they have publishers and news organizations go after them.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#219

Post by HarHarVeryFunny »

simonw wrote: post #213
HarHarVeryFunny wrote: post #211

Earlier quoted context omitted.

I'm not sure what you're getting at. What's useful about LLMs, and especially multi-modal ones, is that that you can ask them anything and they'll answer to best of their ability (especially if well prompted). I'm not sure that o3, as a "reasoning" model is adding much value here - since there is not a whole lot of reasoning going on. This is basically fine-grained image captioning followed by nearest neighbor search…

How does nearest neighbor search relate to this?

If you forget the LLM implementation, fundamentally what you are trying to do here is first detect a bunch of features in the photo (i.e. fine-grain image captioning "in foreground a firepit with safety warning on glass, in background a model XX car parked in front of a bungalow, in distance rolling hills" etc) then do a fuzzy match of this feature set with other photos you have seen - which ones have the greatest number of things in common to the photo you are looking up? You could implement this in a custom app by creating a high-dimensional feature space embedding then looking for nearest neighbors, similar to how face recognition works.

Of course an LLM is performing this a bit differently, and with a bit more flexibility, but the starting point is going to be the same - image feature/caption extraction, which in combination then recall related training samples (both text-only, and perhaps multi-model) which are used to predict the location answer you have asked for. The flexibility of the LLM is that it isn't just treating each feature ("fire pit", "CA licence plate") as independent, but will naturally recall contexts where multiple of these occur together, but IMO not so different in that regard to high dimensional nearest neighbor search.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#220

Post by kevinventullo »

robbie-c wrote: post #159
neurostimulant wrote: post #141

Earlier quoted context omitted.

> when I asked it how, it mentioned that it knows I live nearby. > The process for how it arrives at the conclusion is somewhat similar to humans. It looks at vegetation, terrain, architecture, road infrastructure, signage, and it just knows seemingly everything about all of them. Can we trust what the model says when we ask it about how it comes up with an answer?

Probably not, see https://www.anthropic.com/research/reasoning-models-dont-say...

Would be interesting to apply Interpretability techniques in order to understand how the model really reasons about it.
Post reply on HN