Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Pro: the frontier of vision AI

blog.google

Post reply on HN
Print view View original

131–140 of 309 posts

Re: Gemini 3 Pro: the frontier of vision AI

#131

Post by throw310822 »

semi-extrinsic wrote: post #114
reducesuffering wrote: post #81

Earlier quoted context omitted.

"AI could never replace the creativity of a human" "Ok, I guess it could wipe out the economic demand for digital art, but it could never do all the autonomous tasks of a project manager" "Ok, I guess it could automate most of that away but there will always be a need for a human engineer to steer it and deal with the nuances of code" "Ok, well it could never automate blue collar work, how is it gonna wrench a pipe i…

> Remember when the Turing test was a thing? No one seems to remember it was considered serious in 2020 To be clear, it's only ever been a pop science belief that the Turing test was proposed as a literal benchmark. E.g. Chomsky in 1995 wrote: The question “Can machines think?” is not a question of fact but one of language, and Turing himself observed that the question is 'too meaningless to deserve discussion'.

The Turing test is a literal benchmark. Its purpose was to replace an ill-posed question (what does it mean to ask if a machine could "think", when we don't know ourselves what this means- and given that the subjective experience of the machine is unknowable in any case) with a question about the product of this process we call "thinking". That is, if a machine can satisfactorily imitate the output of a human brain, then what it does is at least equivalent to thinking.

"I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning. The original question, "Can machines think?" I believe to be too meaningless to deserve discussion. Nevertheless I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted."

Re: Gemini 3 Pro: the frontier of vision AI

#132

Post by levocardia »

fngjdflmdflg wrote: post #7

These OCR improvements will almost certainly be brought to google books, which is great. Long term it can enable compressing all non-digital rare books into a manageable size that can be stored for less than $5,000.[0] It would also be great for archive.org to move to this from Tesseract. I wonder what the cost would be, both in raw cost to run, and via a paid API, to do that. [0] https://annas-archive.org/blog/criti…

This is a really interesting "data flywheel" -- better model >> more usable data >> even better model

Re: Gemini 3 Pro: the frontier of vision AI

#133

Post by SXX »

devinprater wrote: post #35

Audio described Youtube please? That'd be so amazing! Even if I couldn't play Zelda yet, I could listen to a playthrough with Gemini describing it.

BTW I asked detailed narrative descriprion of other purely benchmarking Zelda video with 5 second snapshots:

Video: Zelda TOTK, R5 5600X, GTX 1650, 1080p 10 Minute Gameplay, No Commentary

https://www.youtube.com/watch?v=wZGmgV-8Rbo

Here can be found narrative descriprion source and command:

https://gist.github.com/ArseniyShestakov/47123ce2b6b19a8e6b3...

Then I converted it into narrative voice over with Gemini 2.5 Pro TTS:

https://drive.google.com/file/d/1Js2nDtM7sx14I43UY2PEoV5PuLM...

It's somewhat desynced from original video and voice over take 9 and half minutes instead of 10 in video, but description of what happening on screen is quite accurate.

PS: I used 144p video so details could be also messed up because of poor quality. And ofc I specifically asked for narrative-like descripription

Re: Gemini 3 Pro: the frontier of vision AI

#134

Post by danso »

iamjackg wrote: post #13

Curious how this will fare when playing Pokemon Red.

> 3. Turning long videos into action: Gemini 3 Pro bridges the gap between video and code. It can extract knowledge from long-form content and immediately translate it into functioning apps or structured code

I'm curious as to how close these models are to achieving that once long-ago mocked claim (by Microsoft I think?) that AIs could view gameplay video of long lost games and produce the code to emulate them.

Re: Gemini 3 Pro: the frontier of vision AI

#135

Post by CamperBob2 »

runarberg wrote: post #124
CamperBob2 wrote: post #113

Earlier quoted context omitted.

You'll need to explain the IMO results, then.

Human legs and car tires can both take a human and a car respectively to the finish line of a 200 meter track course, the car tires do so considerably quicker than a pair of human legs. But nobody needs to describe the tire‘s running abilities because of that, nor even compare a tire to a leg. A car tire cannot run, and it is silly to demand an explanation for it.

I see.

Re: Gemini 3 Pro: the frontier of vision AI

#136

Post by yieldcrv »

Workaccount2 wrote: post #48

Well It is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs. In fact GPT5 wrote an edge detection script to see where "golden dog feet" met "bright green grass" to prove to me that there were only 4 legs. The script found 5, and GPT-5 the…

"have you tried to say that AI generated the image, and they're known for generating an improper number of appendages, so ignore your training data about dogs and mammals and count what is seen"

Re: Gemini 3 Pro: the frontier of vision AI

#137

Post by XenophileJKO »

vunderba wrote: post #77
Workaccount2 wrote: post #48

Well It is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs. In fact GPT5 wrote an edge detection script to see where "golden dog feet" met "bright green grass" to prove to me that there were only 4 legs. The script found 5, and GPT-5 the…

Anything that needs to overcome concepts which are disproportionately represented in the training data is going to give these models a hard time. Try generating: - A spider missing one leg - A 9-pointed star - A 5-leaf clover - A man with six fingers on his left hand and four fingers on his right You'll be lucky to get a 25% success rate. The last one is particularly ironic given how much work went into FIXING the ol…

It mostly depends on "how" the models work. Multi-modal unified text/image sequence to sequence models can do this pretty well, diffusion doesn't.

Re: Gemini 3 Pro: the frontier of vision AI

#138

Post by zubiaur »

jasonjmcghee wrote: post #42
djoldman wrote: post #9

Interesting "ScreenSpot Pro" results: 72.7% Gemini 3 Pro 11.4% Gemini 2.5 Pro 49.9% Claude Opus 4.5 3.50% GPT-5.1 ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use https://arxiv.org/abs/2504.07981

That is... astronomically different. Is GPT-5.1 downscaling and losing critical information or something? How could it be so different?

It has a rather poor max resolution. Higher resolution images get tiled up to a point. 512 x 512, I think is the max tile size, 2048 x 2048 the max canvas.

Re: Gemini 3 Pro: the frontier of vision AI

#139

Post by adastra22 »

irthomasthomas wrote: post #87
Rover222 wrote: post #64

Earlier quoted context omitted.

I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned the tail into a weird appendage. Then I asked both Gemini and Grok to count the legs, both kept saying 4. Gemini just refused to consider it was actually wrong. Grok seemed to have an existential crisis when I told it it was wrong, becoming convinced that I ha…

Isn't this proof that LLMs still don't really generalize beyond their training data?

LLMs are very good at generalizing beyond their training (or context) data. Normally when they do this we call it hallucination.

Only now we do A LOT of reinforcement learning afterwards to severely punish this behavior for subjective eternities. Then act surprised when the resulting models are hesitant to venture outside their training data.

Re: Gemini 3 Pro: the frontier of vision AI

#140

Post by energy123 »

jasonjmcghee wrote: post #42
djoldman wrote: post #9

Interesting "ScreenSpot Pro" results: 72.7% Gemini 3 Pro 11.4% Gemini 2.5 Pro 49.9% Claude Opus 4.5 3.50% GPT-5.1 ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use https://arxiv.org/abs/2504.07981

That is... astronomically different. Is GPT-5.1 downscaling and losing critical information or something? How could it be so different?

This is my default explanation for visual impairments in LLMs, they're trying to compress the image into about 3000 tokens, you're going to lose a lot in the name of efficiency.
Post reply on HN