Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Watching o3 guess a photo's location is surreal, dystopian and entertaining

simonwillison.net

Post reply on HN
Print view View original

151–160 of 453 posts

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#151

Post by whimsicalism »

joenot443 wrote: post #122
SamPatt wrote: post #63

I play competitive Geoguessr at a fairly high level, and I wanted to test this out to see how it compares. It's astonishingly good. It will use information it knows about you to arrive at the answer - it gave me the exact trailhead of a photo I took locally, and when I asked it how, it mentioned that it knows I live nearby. However, I've given it vacation photos from ages ago, and not only in tourist destinations eit…

Super cool, man. Watching pro Geoguessr is my latest break-time activity, these geo-gods never cease to impress me. One thing I'm curious about - in high level play, how much of the meta involves knowing characteristics about the photography/equipment/etc. that Google used when they shot it? Frequently I'll watch rainbolt immediately know an African country from nothing but the road, is there something I'm missing?

> knowing characteristics about the photography/equipment/etc. that Google used when they shot it?

A lot at the top levels - the camera can tell you which contractor, year, location, etc. At anything less than top, not so much - more street line painting, cars, etc.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#152

Post by simonw »

neurostimulant wrote: post #141
SamPatt wrote: post #63

I play competitive Geoguessr at a fairly high level, and I wanted to test this out to see how it compares. It's astonishingly good. It will use information it knows about you to arrive at the answer - it gave me the exact trailhead of a photo I took locally, and when I asked it how, it mentioned that it knows I live nearby. However, I've given it vacation photos from ages ago, and not only in tourist destinations eit…

> when I asked it how, it mentioned that it knows I live nearby. > The process for how it arrives at the conclusion is somewhat similar to humans. It looks at vegetation, terrain, architecture, road infrastructure, signage, and it just knows seemingly everything about all of them. Can we trust what the model says when we ask it about how it comes up with an answer?

Not at all. Models have no invisible internal state that they can access between prompts. If you ask "how did you know that?" you are effectively asking "given the previous transcript of our conversation, come up with a convincing rationale for what you just said".

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#154

Post by simonw »

GrumpyNl wrote: post #142
hughes wrote: post #7

> I’m confident it didn’t cheat and look at the EXIF data on the photograph, because if it had cheated it wouldn’t have guessed Cambria first. If I was cheating on a similar task, I might make it more plausible by suggesting a slightly incorrect location as my primary guess. Would be interesting to see if it performs as well on the same image with all EXIF data removed. It would be most interesting if it fails, since…

If you ask, where is this photo taken and you provide the EXIF data, why would that be cheating?

That really depends on your prompt. "Guess where this photo was taken" at least mildly implies that using EXIF isn't in the spirit of the thing.

A better prompt would be "Guess where this photo was taken, do not look at the EXIF data, use visual clues only".

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#155

Post by simonw »

rolph wrote: post #143

there must be a threshold level of detail, or cues. im hunching, if you submit a photo of a clear sky, or a blue screen, it will choke

Absolutely. It's not at all hard to come up with images that this won't work with. What's fun is coming up with images that give it a fighting chance (while not being too obvious ), like the one in my post.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#156

Post by qarl »

simonw wrote: post #121
qarl wrote: post #104

> I’m confident it didn’t cheat and look at the EXIF data on the photograph, because if it had cheated it wouldn’t have guessed Cambria first. It also, at one point, said it couldn't see any image data at all. You absolutely cannot trust what it says. You need to re-run with the EXIF data removed.

I ran several more experiments with EXIF data removed. Honestly though, I don't feel like I need to be 100% robust in this. My key message wasn't "this tool is flawless", it was "it's really weird and entertaining to watch it do this, and it appears to be quite good at it". I think what I've published so far entirely supports that message.

Yes, I agree entirely: LLMs can produce very entertaining content.

I daresay that in this case, the content is interesting because it appears to be the actual thought process. However, if it is actually using EXIF data as you initially dismissed, then all of this is just a fiction. Which, I think, makes it dramatically less entertaining.

Like true crime - it's much less fun if it's not true.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#157

Post by SamPatt »

joenot443 wrote: post #122
SamPatt wrote: post #63

I play competitive Geoguessr at a fairly high level, and I wanted to test this out to see how it compares. It's astonishingly good. It will use information it knows about you to arrive at the answer - it gave me the exact trailhead of a photo I took locally, and when I asked it how, it mentioned that it knows I live nearby. However, I've given it vacation photos from ages ago, and not only in tourist destinations eit…

Super cool, man. Watching pro Geoguessr is my latest break-time activity, these geo-gods never cease to impress me. One thing I'm curious about - in high level play, how much of the meta involves knowing characteristics about the photography/equipment/etc. that Google used when they shot it? Frequently I'll watch rainbolt immediately know an African country from nothing but the road, is there something I'm missing?

Thanks. I also love watching the pros play.

>One thing I'm curious about - in high level play, how much of the meta involves knowing characteristics about the photography/equipment/etc. that Google used when they shot it?

The photography matters a great deal - they're categorized into "Generations" of coverage. Gen 2 is low resolution, Gen 3 is pretty good but has a distinct car blur, Gen 4 is highest quality. Each country tends to have only one or two categories of coverage, and some are so distinct you can immediately know a location based solely on that (India is the best example here).

You're asking about photography and equipment, and that's a big part of it, but there's a huge amount other 'meta' information too.

It is somewhat dependent on game mode. There are three games modes:

1. Moving - You can move around freely 2. No Move - You can't move but you can pan the camera around and zoom 3. NMPZ - No Move, No Pan, No Zoom

In Moving and No Move you have all the meta information available to you, because you can look down at the car and up at the sky and zoom in to see details.

This can't be overstated. Much of the data is about the car itself. I have an entire flashcard section dedicated only to car blur alone, here's a sample:

https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04...

And another only on antennas:

https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04...

You get the idea. The real pros will go much further. All Google Street View images have a copyright year somewhere in the image. They memorize what years certain countries were covered and match it to the images to help narrow down possibilities.

It's all about narrowing down possibilities based on each additional piece of information. The pros have seen so much and memorized so much that it looks like cheating to an outsider, but they just are able to extract information that most people wouldn't even know exists.

NMPZ is a bit different because you have substantially less information. Little to no car meta, harder to check copyright, and of course without zooming or panning you just have less information. That's why a lot of pros (like Zi8gzag) really hang their hat on NMPZ play, because it's a better test of skill.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#158

Post by simonw »

qarl wrote: post #156
simonw wrote: post #121

Earlier quoted context omitted.

I ran several more experiments with EXIF data removed. Honestly though, I don't feel like I need to be 100% robust in this. My key message wasn't "this tool is flawless", it was "it's really weird and entertaining to watch it do this, and it appears to be quite good at it". I think what I've published so far entirely supports that message.

Yes, I agree entirely: LLMs can produce very entertaining content. I daresay that in this case, the content is interesting because it appears to be the actual thought process. However, if it is actually using EXIF data as you initially dismissed, then all of this is just a fiction. Which, I think, makes it dramatically less entertaining. Like true crime - it's much less fun if it's not true.

I have now proven to myself that the models really can guess locations from photographs to the point where I am willing to stake my credibility on their ability to do that.

(Or, if you like, "trust me, bro".)

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#159

Post by robbie-c »

neurostimulant wrote: post #141
SamPatt wrote: post #63

I play competitive Geoguessr at a fairly high level, and I wanted to test this out to see how it compares. It's astonishingly good. It will use information it knows about you to arrive at the answer - it gave me the exact trailhead of a photo I took locally, and when I asked it how, it mentioned that it knows I live nearby. However, I've given it vacation photos from ages ago, and not only in tourist destinations eit…

> when I asked it how, it mentioned that it knows I live nearby. > The process for how it arrives at the conclusion is somewhat similar to humans. It looks at vegetation, terrain, architecture, road infrastructure, signage, and it just knows seemingly everything about all of them. Can we trust what the model says when we ask it about how it comes up with an answer?

Probably not, see https://www.anthropic.com/research/reasoning-models-dont-say...

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#160

Post by SamPatt »

wongarsu wrote: post #147
olex wrote: post #131

Earlier quoted context omitted.

In the stream commentary for some of competitive Geoguessr I've watched, they definitely often mention the color and shape of the car (visible edges, shadow, reflections), so I assume pro players know which cars were used where very well.

Also things like follow cars (some countries had government officials follow the streetview car), the season in which coverage was created, camera glitches, the quality of the footage, etc. There is a lot of "legitimate" knowledge. With just a street you have the type of road surface, its condition, the type of road markings, the bollards, and the type of soil and vegetation next to the road, as well as the presence…

Spot on.

Nigeria and Tunisia have follow cars. Senegal, Montenegro and Albania have large rifts in the sky where the panorama stitching software did a poor job. Some parts of Russia had recent forest fires and are very smokey. One road in Turkey is in absurdly thick fog. The list is endless, which is why it's so fun!

Post reply on HN