Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Watching o3 guess a photo's location is surreal, dystopian and entertaining

simonwillison.net

Post reply on HN
Print view View original

361–370 of 453 posts

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#361

Post by TeMPOraL »

roarcher wrote: post #312
Timwi wrote: post #291

Earlier quoted context omitted.

> Give it something weird and it'll continuously trip over itself. And so will almost all humans. It's weird how people refuse to ascribe any human-level intelligence to it until it starts to compete with the world top elite.

Yeah, but humans can be made to understand when and how they're wrong and narrow their focus to fixing the mistake. LLMs apologize and then proudly present the exact same output as before, repeatedly, forever spinning their wheels at the first major obstacle to their reasoning.

> LLMs apologize and then proudly present the exact same output as before, repeatedly, forever spinning their wheels at the first major obstacle to their reasoning.

So basically like a human, at least up to young adult years in teaching context[0], where the student is subject to authority of the teacher (parent, tutor, schoolteacher) and can't easily weasel out of the entire exercise. Yes, even young adults will get stuck in a loop, presenting "the exact same output as before, repeatedly, forever spinning their wheels at the first major obstacle to their reasoning", or at least until something clicks, or they give up in shame (or the teacher does).

--

[0] - Which is where I saw this first-hand.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#362

Post by 0x008 »

c7b wrote: post #350

> Technology can identify locations from photographs now. Tbf, it could do that before, and probably still better than the LLM: https://youtube.com/watch?v=ts5lPDV--cU But seeing it as what appears to be an emergent capability in such a general model is something else.

thanks for posting this video, that was really entertaining!

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#363

Post by iamkd »

qarl wrote: post #104

> I’m confident it didn’t cheat and look at the EXIF data on the photograph, because if it had cheated it wouldn’t have guessed Cambria first. It also, at one point, said it couldn't see any image data at all. You absolutely cannot trust what it says. You need to re-run with the EXIF data removed.

I have been regularly testing o3 in terms of geoguessing, and the first thing it usually does is run a Python script that extracts EXIF. So definitely could be the case

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#364

Post by xlii »

SamPatt wrote: post #292
xlii wrote: post #209

Tried the same, results made me laugh. Completely clueless. I've seen passing prompts 8 about how it's not in the city I am and yet it tries again and again. My favourite moment was when it started analysing piece of blurry asphalt. After 6 minutes o3 it was confidently wrong: https://imgur.com/a/jYr1fz1 IMO not-in-US is actually great test if something was in LLMs data and the whole search is a for show.

I'm surprised to hear that. I keep running tests and the results are incredible, not only in the US. For example, here's a screenshot from a random location I found in Google Street View in Jordan: https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04... And here's o3 nailing it: https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04... Maybe using Google Street View images, zoomed out, tends to give mor…

IMO the „thought” process is completely fake.

I wanted o3 to succeed so I gave more and more details. Every attempt was approx. 8 minute and it took 1h in total.

The extra input I provided (in order):

- belt of location of width of 40km (results and searches were made outside of the range)

- explicitly stated cities to omit (ignored instruction)

- construction date (wasn’t used in searches)

- OSM amenity (townhall) - streetnumber (it insisted that it’s incorrect and keep giving other result) - at that point there were only 6 results from overpass

- another photo with actual partial name of the city

- 8 minutes later it correlated it found using flag colors in front of the building

As others stated this „thought” process is completely hallucinating. IMO you either fall into bucket or good luck finding it.

On the other hand I decided to tryout Gemini for some personal project and I found responses much better than GPTs. Not about correctness but in „attitude” form.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#365

Post by nurettin »

AlexCoventry wrote: post #340
nurettin wrote: post #326

Earlier quoted context omitted.

If you'd like a creative waste of time, make it implement any novel algorithm that mixes the idea of X with Y. It will fail miserably, double down on the failure and hard troll you, run out of context and leave you questioning why you even pay for this thing. And it is not something that can be fixed with more specific training.

Can you give an example? Have you tried it recently with the higher-end models?

My favorite example is implementing NEAT with keras dense layers instead of graphs. Last time I tried with claude 3.7, it wrote code to mutate the output layer (??). I tried to prevent that a few times and gave up.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#366

Post by ghaff »

SamPatt wrote: post #276
matthewdgreen wrote: post #272

Earlier quoted context omitted.

I was absolutely gobsmacked by the three minute chain of reasoning this thing did, and how it absolutely nailed the location of the photo based on plants, the color of a fence, comparison with nearby photos, and oh yeah, also the EXIF data containing the exact lat/long coordinates that I accidentally left in the file. https://bsky.app/profile/matthewdgreen.bsky.social/post/3lnq...

Lol it's very easy to give the models what they need to cheat. For my test I used screenshots to ensure no metadata. I mentioned this in another comment but I was a part of an AI safety fellowship last year where we created a benchmark for LLMs ability to geolocate. The models were doing unbelievably well, even the bad open source ones, until we realized our image pipeline was including location data in the filename!…

I was and am pretty impressed by Google Photo/Lens IDs. But I realized fairly early on that of course it knew the locations of my iPhone photos from the geo info stored in the photo.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#367

Post by jitl »

josephg wrote: post #335
jdiff wrote: post #259

Earlier quoted context omitted.

Definitely matches my experience as well. I've been working away on a very quirky, non-idiomatic 3D codebase, and LLMs are a mixed bag there. Y is down, there's no perspective distortion or Z buffer, there are no meshes, it's a weird place. It's still useful to save me from writing 12 variations of x1 = sin(r2) - cos(r1) while implementing some geometric formula, but absolutely awful at understanding how those fit in…

Yeah I have the same experience. I’ve done some work on novel realtime text collaboration algorithms. For optimisation, I use some somewhat bespoke data structures. (Eg I’m using an order-statistic tree storing substring lengths with internal run-length encoding in the leaf nodes). ChatGPT is pretty useless with this kind of code. I got it to help translate a run length encoded b-tree from rust to typescript. Even wi…

It’s just not there yet but I think it will get there for translation kind of tasks quite capably in the next 12 months, especially if asked to translate a single file or a selection in a file line by line. Right now it’s quite bad which I find surprising. I have less confidence we’ll see whole-codebase or even module level understanding for novel topics in the next 24 months.

There’s also a question of quality of source data. At least in TypeScript/JavaScript land, the vast majority of code appears to be low quality and buggy or ignores important edge cases and so even when working on “boilerplate” it can produce code that appears to work but will fall over in production for 20% of users (for example string handling code that will tear Unicode graphemes like emoji).

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#368

Post by larodi »

SamPatt wrote: post #63

I play competitive Geoguessr at a fairly high level, and I wanted to test this out to see how it compares. It's astonishingly good. It will use information it knows about you to arrive at the answer - it gave me the exact trailhead of a photo I took locally, and when I asked it how, it mentioned that it knows I live nearby. However, I've given it vacation photos from ages ago, and not only in tourist destinations eit…

Is it meaningful to conclude that this is an algorithm that pro GGsrs all follow, and one of them perhaps explained somewhere and the model took it? Is geo-guessing something that can be presented as algorithm or steps? Perhaps it is not as challenging as it seems, given one knows what to look for?

not as challenging... as say complex differential geometry.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#369

Post by bonoboTP »

oceanplexian wrote: post #358
spaceman_2020 wrote: post #342

Earlier quoted context omitted.

This is also why I buy the apocalyptic headlines about AI replacing white collar labor - most white collar employment is mostly creating the same things (a CRUD app, a landing page, a business plan) with a few custom changes Not a lot of labor is actually engaged in creating novel things. The marketing plan for your small business is going to be the same as the marketing plan for every other small business with some…

I agree but the reason it won’t be an apocalypse is the same reason economists get most things wrong, it’s not an efficient market. Relatively speaking we live in a bubble, there are still broad swaths of the economy that operate with pen and paper. Another broad swath that migrated off 1980s era AS/400 in the last few years. Even if we had ASI available literally today (And we don’t) I’d give it 20-30 years until th…

I had predicted the same about websites, social media presence, Google maps presence etc. back 10-15 years ago, but lo and behold, even the small burger place hole-on-a-wall in rural eastern Europe is now on Google maps with reviews, and even answers by the owner, a facebook page with info on changes of opening hours etc. I'd have said there's no way that fat 60 year old guy will get up to date with online stuff.

But gradually they were forced to.

If there are enough auto repair shops that can just diagnose and process n times more cars in a day, it will absolutely force people to adopt it as well, whether they like the aesthetics or not, whether they feel like learning new things or not. Suddenly they will be super interested in how to use it, regardless of how they were boasting about being old school and hands-on beforehand.

If a technology gives enough boost to productivity, there's simply no way for inertia to hold it back, outside of the most strictly regulated fields, such as medicine, which I do expect to lag behind by some years, but will have to catch up once the benefits are clear in lower-stakes industries and there's immense demand on it that politicians will be forced to crush the doctor's cartel's grip on things.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#370

Post by casey2 »

cyral wrote: post #338
simonw wrote: post #95

Earlier quoted context omitted.

If you want to be impressed I suggest trying this yourself on your own photos. I don't consider it my job to impress or mind-blow people: I try to present as realistic as possible a representation of what this stuff can do. That's why I picked an example where its first guess was 200 miles off!

Reading the replies to this is funny. It's like the classic dropbox thread. "But this could be done with a nearest neighbor search and feature detection!" If this isn't mind blowing to someone I don't know if any amount of explaining will help them get it.

It's not mindblowing because there were public systems doing performing much better years earlier. Using the exact same tech. This is less like rsync vs drop box and more like you are freaking out over Origin or Uplay when Steam has been around for years.
Post reply on HN