Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Watching o3 guess a photo's location is surreal, dystopian and entertaining

simonwillison.net

Post reply on HN
Print view View original

421–430 of 453 posts

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#421

Post by busyant »

iamkd wrote: post #363
qarl wrote: post #104

> I’m confident it didn’t cheat and look at the EXIF data on the photograph, because if it had cheated it wouldn’t have guessed Cambria first. It also, at one point, said it couldn't see any image data at all. You absolutely cannot trust what it says. You need to re-run with the EXIF data removed.

I have been regularly testing o3 in terms of geoguessing, and the first thing it usually does is run a Python script that extracts EXIF. So definitely could be the case

I took screenshots of existing 20 year old digital photos ... so ... no relevant exif data.

o3 was quite good at locating, even when I gave it pics with no discernible landmarks. It seemed to work off of just about anything it could discern from the images:

* color of soil

* type of telephone pole

* type of bus stop

* tree types, tree sizes, tree ages, etc.

* type of grass. etc.

It got within a 50 mile radius on the two screenshots I uploaded that had no landmarks.

If I uploaded pics with discernible landmarks (e.g., distant hill, etc.), it got within ~ 20 mile radius.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#422

Post by dbdoskey »

flir wrote: post #396
coffeebeqn wrote: post #357

Earlier quoted context omitted.

I don’t know if most software engineers build toy CRUD apps all day? I have found the state of the art models to be almost completely useless in a real large codebase. Tried Claude and Gemini latest since the company provides them but they couldn’t even write tests that pass after over a day of trying

Our current architectures are complex, mostly because of DRY and a natural human tendency to abstract things. But that's a decision, not a fundamental property of code. At core, most web stuff is "take it out of the database, put it on the screen. Accept it from the user, put it in the database." If everything was written PHP3 style (add_item.php, delete_item.php, etc), with minimal includes, a chatbot might be rathe…

This is IMHO where the interesting direction will be. How do we architecture code so that it is optimized around chatbot development? In the past areas of separation were determined by api stability, deployment concerns, or even just internal team politics. In the future a rep might be separated from a monolith repo to be an area of responsibility that a chatbot can reason about, and not get lost in the complexity.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#423

Post by the_duke »

tough wrote: post #416
nradov wrote: post #415

Earlier quoted context omitted.

This might be an argument in favor of a microservices architecture with the code split across many repos rather than a monolithic application with all the code in a single repo. It's not that microservices are necessarily technically better but they could allow you to get more leverage out of LLMs due to context window limitations.

if your microservices become more verbose overall, now you have handicapped your ability to cram the whole codebase into a context window. I think AI is great but humans know the why's of the code needs to exist AI's don't need stuff, only generate it

The LLMs would only need to have API information for the service, not the whole code, which would be small.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#424

Post by simonw »

sgtnoodle wrote: post #412

I'm embarrassed that I live in HMB and didn't recognize it. I should go to the harbor more.

It was at Jettywave Distillery - if you haven't checked it out yet it has fantastic outdoor seating, really good food and the best cocktails in HMB.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#425

Post by svilen_dobrev »

flir wrote: post #396
coffeebeqn wrote: post #357

Earlier quoted context omitted.

I don’t know if most software engineers build toy CRUD apps all day? I have found the state of the art models to be almost completely useless in a real large codebase. Tried Claude and Gemini latest since the company provides them but they couldn’t even write tests that pass after over a day of trying

Our current architectures are complex, mostly because of DRY and a natural human tendency to abstract things. But that's a decision, not a fundamental property of code. At core, most web stuff is "take it out of the database, put it on the screen. Accept it from the user, put it in the database." If everything was written PHP3 style (add_item.php, delete_item.php, etc), with minimal includes, a chatbot might be rathe…

long time ago, in one small company, i wrote an accounting system from 1st principles and then it was deployed to some large-ish client. It took several months of rearranging their whole workflows and quarelling with their operators to enable the machine to do what it is good at and to disable all the human-related quirky +optimizations -cover-asses. Like, humans are good at rough guessing but bad at remembering/repeating same thing. Hence usual manual accounting workflows are heavily optimized for error-avoidability.

Seems same thing here.. another kind of bitter lesson, maybe less bitter :/

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#426

Post by forgotTheLast »

bonoboTP wrote: post #403
forgotTheLast wrote: post #245

I tried it twice with 4o and the results were comical: - picture taken on a road through a wooded park: It correctly guessed north america based on vegetation. Then incorrectly guessed Minnesota based on the type of fence. I tried to steer it in the right direction by pointing out license plates and signage but it then hallucinated a front license plate from Ontario on a car that didn't have any, then hallucinated a…

4o is not a "reasoning" model. o3 and o4 are.

Yes my mistake, it was o4-mini.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#427

Post by amazingman »

I'm trying this now with a difficult example and it's spinning. A lot. It's really interesting to see the methodology of its approach, but I am slowly becoming certain that this single chat costs more than every other chat I've ever had. Maybe even more than this month's $20. It's just spiraling.

(It eventually "lost" the network connection)

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#429

Post by DidYaWipe »

simonw wrote: post #266
DidYaWipe wrote: post #264

So... where do you go to try this? I didn't notice any link in the article.

https://chatgpt.com - I was using o3 which I think is paid only, but o4-mini and o4-mini-high should both provide similar results and I think at least one of those is available on the free plan. EDIT: My mistake, looks like those models are only available on the $20/month Plus plan or higher. I added a note about that to my post.

Thanks for the info. Do they still require a working phone number? Their insistence on that is why I have never tried ChatGPT.

Re: Watching o3 guess a photo's location is surreal, dystopian and entertaining

#430

Post by simonw »

DidYaWipe wrote: post #429
simonw wrote: post #266

Earlier quoted context omitted.

https://chatgpt.com - I was using o3 which I think is paid only, but o4-mini and o4-mini-high should both provide similar results and I think at least one of those is available on the free plan. EDIT: My mistake, looks like those models are only available on the $20/month Plus plan or higher. I added a note about that to my post.

Thanks for the info. Do they still require a working phone number? Their insistence on that is why I have never tried ChatGPT.

I imagine they do, because that's a useful way to prevent people from signing up for hundreds of accounts to abuse a free preview tier. Phone numbers are harder to obtain many of them other things like email addresses.

I've never heard of them abusing that phone number for anything other than that initial verification code.

Post reply on HN