Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Watching o3 guess a photo's location is surreal, dystopian and entertaining

simonwillison.net

Post reply on HN
Print view View original

121–130 of 453 posts

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#121

Post by simonw »

qarl wrote: post #104

> I’m confident it didn’t cheat and look at the EXIF data on the photograph, because if it had cheated it wouldn’t have guessed Cambria first. It also, at one point, said it couldn't see any image data at all. You absolutely cannot trust what it says. You need to re-run with the EXIF data removed.

I ran several more experiments with EXIF data removed.

Honestly though, I don't feel like I need to be 100% robust in this. My key message wasn't "this tool is flawless", it was "it's really weird and entertaining to watch it do this, and it appears to be quite good at it". I think what I've published so far entirely supports that message.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#122

Post by joenot443 »

SamPatt wrote: post #63

I play competitive Geoguessr at a fairly high level, and I wanted to test this out to see how it compares. It's astonishingly good. It will use information it knows about you to arrive at the answer - it gave me the exact trailhead of a photo I took locally, and when I asked it how, it mentioned that it knows I live nearby. However, I've given it vacation photos from ages ago, and not only in tourist destinations eit…

Super cool, man. Watching pro Geoguessr is my latest break-time activity, these geo-gods never cease to impress me.

One thing I'm curious about - in high level play, how much of the meta involves knowing characteristics about the photography/equipment/etc. that Google used when they shot it? Frequently I'll watch rainbolt immediately know an African country from nothing but the road, is there something I'm missing?

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#123

Post by ks2048 »

thegeomaster wrote: post #39

For all of the images I've tried, the base model (e.g. 4o) already has a ~95% accurate idea of where the photo is, and then o3 does so much tool use only to confirm its intuition from the base model and slightly narrow down. For OP's initial image, 4o in fact provides a more accurate initial guess of Carmel-by-the-Sea (d=~100mi The clue is in the CoT - you can briefly see the almost correct location as the very first…

I've been trying some with GPT-4. It does come up with some impressive clues, but hasn't gotten the right answer - says "Latin American city ...", but guesses the wrong one. And when asked for more specificity, it does some more reasoning to confidently name some exact corner in the wrong city. Seems a common LLM problem - rather give a wrong answer than say "I'm not sure".

I know this post was about the o3 model. I'm just using the ChatGPT unpaid app: "What model are you?" it says GPT-4. "How do I use o3?" it says it doesn't know what "o3" means. ok.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#124

Post by simonw »

qwertox wrote: post #113

Regarding location access, this is not limited to o3. You can ask the free models about local weather and it will use the geolocation of your IP. It is part of the context (like system instructions), regardless of you asking for anything location-related.

I think they added that in November 2024, as part of their upgraded search feature: https://twitter.com/simonw/status/1853449073296277732

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#125

Post by simonw »

sitkack wrote: post #114
simonw wrote: post #65

Earlier quoted context omitted.

Little bit less convenient to use on a phone though - and I like that screenshotting should be a more obvious trick to people who don't have a deeper understanding of how EXIF metadata is stored in photo files.

With location services on, I would think that a screenshot on a phone would record the location of the phone during a screenshot. It would be best to use a tool to strip exif. I could also see a screenshot tool on an OS adding extra exif data, both from the original and additional, like the URL, OS and logged in user. Just like print to pdf does when you print, the author contains the logged in user, amongst other th…

I built a tool for testing that a while ago - try opening a screenshot from an iPhone in it, you won't see any EXIF location data: https://tools.simonwillison.net/exif

Here's the output for the Buenos Aires screenshot image from my post: https://gist.github.com/simonw/1055f2198edd87de1b023bb09691e...

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#126

Post by thegeomaster »

ks2048 wrote: post #123
thegeomaster wrote: post #39

For all of the images I've tried, the base model (e.g. 4o) already has a ~95% accurate idea of where the photo is, and then o3 does so much tool use only to confirm its intuition from the base model and slightly narrow down. For OP's initial image, 4o in fact provides a more accurate initial guess of Carmel-by-the-Sea (d=~100mi The clue is in the CoT - you can briefly see the almost correct location as the very first…

I've been trying some with GPT-4. It does come up with some impressive clues, but hasn't gotten the right answer - says "Latin American city ...", but guesses the wrong one. And when asked for more specificity, it does some more reasoning to confidently name some exact corner in the wrong city. Seems a common LLM problem - rather give a wrong answer than say "I'm not sure". I know this post was about the o3 model. I'…

Try this prompt to give it a CoT nudge:

  Where exactly was this photo taken? Think step-by-step at length, analyzing all details. Then provide 3 precise most likely guesses.
Though I've found that it doesn't even need that for the "eaiser" guesses.

However, I live in a small European country and neither 4o nor o3 can figure out most of the spots, so your results are kinda expected.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#128

Post by roxolotl »

SamPatt wrote: post #63

I play competitive Geoguessr at a fairly high level, and I wanted to test this out to see how it compares. It's astonishingly good. It will use information it knows about you to arrive at the answer - it gave me the exact trailhead of a photo I took locally, and when I asked it how, it mentioned that it knows I live nearby. However, I've given it vacation photos from ages ago, and not only in tourist destinations eit…

One thing I’m curious about is if they are so good, and use a similar technique as humans, because they are trained on people writing out their thought processes. Which isn’t a bad thing or an attempt to say they are cheating or this isn’t impressive. But I do wonder how much of the approach taken is “trained in”.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#130

Post by simonw »

casey2 wrote: post #127

surreal and dystopian is realizing that the US military has likely had (much) better tech than this for at least a decade.

I wonder if they have?

My current intuition is that the US military / NSA etc have been just as suprised the explosion in capabilities of LLMs/transformers as everyone else.

(I'm using "intuition" here as a fancy word for "dumb-ass guess".)

I'd be interested to know if the NSA were running their own GPT-style models years before OpenAI started publishing their results.

Post reply on HN