Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT now performs well at GeoGuesser

flausch.social

Post reply on HN
Print view View original

21–30 of 152 posts

Re: ChatGPT now performs well at GeoGuesser

#21

Post by actuallyalys »

It wouldn’t shock me if multimodal LLMs were good at GeoGuesser [0], but if we’re being picky, it takes more than a few examples to demonstrate a game is “solved.” I also wonder what kind of data leakage might have been at play, like other people have suggested.

To be clear, my point is not that this is unimpressive, just that this doesn’t demonstrate much. (Edit: I should have said, it doesn’t demonstrate what the title claims.)

[0] they were very likely trained on a large number of photos that had their location, and they have the ability to isolate features. Combinined with their ability to interpret instructions and just, well, guess, that seems like you have enough for the game.

Re: ChatGPT now performs well at GeoGuesser

#23

Post by bee_rider »

xnickb wrote: post #19
bee_rider wrote: post #16

Earlier quoted context omitted.

Of course, it is pretty impressive. It isn’t magic. I think there we’ll often be surprised by the “hits,” but it’ll also get a lot of “misses.” A random photo in your back yard that doesn’t show the horizon will probably be a miss, right? Meanwhile, lots of photos already include GPS coordinates… data you can strip out, but which is enabled by default for millions of people. Privacy, eh, never knew it I guess.

It's a fallacy to assume that one didn't have privacy merely because most people opted out of it due to personal reasons.

Which fallacy is it, out of curiosity?

Re: ChatGPT now performs well at GeoGuesser

#24

Post by Mistletoe »

colordrops wrote: post #20
xeonmc wrote: post #2

The steam engine was invented and the 100m dash is now a solved problem.

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra

Isn’t a human brain an organic computer? I find the human brain very interesting. The most interesting thing I know of in the universe.

Re: ChatGPT now performs well at GeoGuesser

#25

Post by MattGaiser »

dataviz1000 wrote: post #12
Retr0id wrote: post #9

In all these examples, I wonder if it's indirectly able to draw on the user's own location? Not necessarily via image metadata, but the request origin IP etc. If I ask ChatGPT for the weather forecast, I get it for my own location. Would be interesting to have someone reproduce coming from a different country.

I'm in Lima, Peru on vacation. Yes, it knows where I am.

I am sending it screenshots of my vacation pictures (i.e. without the metadata) and screenshots from GeoGuessr. It’s not doing well and is making guesses based on my location.

So I sent it a photo from Warsaw and it guessed a place in my hometown of Calgary.

Sent it a picture of Bangkok and it guessed Calgary in the summer or Hawaii or something like Singapore.

It confidently stated that a photo from Versailles was Hyde Park.

It correctly identified that the panda photo was from the zoo, but said it exactly matched a zoo in Sichuan, a place I have never been.

It identified a street as being in Bridgewater Nova Scotia, when it was actually in Miramachi NB. Even made up business names for the buildings.

It identified a random house in Moncton as the "old sheriffs house in Annapolis Royal, Nova Scotia.

A beach in Phuket was identified as Destin, Florida.

Re: ChatGPT now performs well at GeoGuesser

#26

Post by layman51 »

xeonmc wrote: post #2

The steam engine was invented and the 100m dash is now a solved problem.

GeoGuessr is also not a “solved problem” in the sense that if you give the model a photo of an outdoor location that is not covered by Google Street View, then it will just make an educated guess which might still be many kilometers away.

Re: ChatGPT now performs well at GeoGuesser

#28

Post by omalled »

jsheard wrote: post #11
romanhn wrote: post #4

The Alki Beach example is absolute madness. On one hand, I can't wait until all thousands of my photos get automatic semantic and geographic tagging (I guess that's possible now). On the other, goodbye privacy, we hardly knew ye. It will be interesting to apply this to historical, or just old, photos.

> The Alki Beach example is absolute madness. I wonder about info leakage with that one, the poster uses that exact photo as their avatar so if they've mentioned Alki Beach before then reverse image searching might pick it up from context. Ideally you'd want to test it with a photo that's never been posted online.

I also wonder how much the memory feature plays a role. I gave it a picture of the street I live on that isn’t online anywhere. O3 guessed the town correctly. Then, I turned off the memory feature and did the same thing. It didn’t even get the state right.

Re: ChatGPT now performs well at GeoGuesser

#29

Post by xnickb »

dredmorbius wrote: post #15
xeonmc wrote: post #2

The steam engine was invented and the 100m dash is now a solved problem.

A far more apropos comparison would be the internal combustion engine and the horse, in a military context. Though sticking with steam engines, military logistics advantaged over a wagon caravan. The question here isn't a casual guessing game, but threat models (as directly addressed in TFA), and general informational hygiene.

I fail to see the drastic change here. AI was used for this in the past as well. The difference is far from that between steam and combustion engines.

Re: ChatGPT now performs well at GeoGuesser

#30

Post by echelon »

actuallyalys wrote: post #21

It wouldn’t shock me if multimodal LLMs were good at GeoGuesser [0], but if we’re being picky, it takes more than a few examples to demonstrate a game is “solved.” I also wonder what kind of data leakage might have been at play, like other people have suggested. To be clear, my point is not that this is unimpressive, just that this doesn’t demonstrate much. (Edit: I should have said, it doesn’t demonstrate what the t…

I was thinking it was using IP geolocation, but after experimenting, I think it's just generally informed.

Here are a few results from GPT 4.5:

https://imgur.com/a/lGTipnn

Post reply on HN