Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Pro: the frontier of vision AI

blog.google

Post reply on HN
Print view View original

121–130 of 309 posts

Re: Gemini 3 Pro: the frontier of vision AI

#121

Post by ActorNightly »

pwagland wrote: post #74
danielvaughn wrote: post #57

Earlier quoted context omitted.

I don’t know much about AI, but I have this image test that everything has failed at. You basically just present an image of a maze and ask the LLM to draw a line through the most optimal path. Here’s how Nano Banana fared: https://x.com/danielvaughn/status/1971640520176029704?s=46

I tried this with Claude: ``` > [Image #1] Create a unicode "ascii-art" version of this image, with the optimal path through the maze highlighted in a solid colour. I'll create an ASCII art version of this maze with the solution path highlighted! ┌─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┐ ●●│ │ │ │ │ │ │ │ │ │ │ │ │ │ ├─┤●└─┴─┐ ├───┐ │ ╔═╗ ├─────┤ ╔═══╝ │ ╔═╝ ╔═╝ │ │ ╔═╝ ├─┤ │ │●●●●●└─┤ │ │ ║ │ │ │ ║…

>Suffice to say, it didn't do either part right.

I dunno why people are surprised by this. This is what you get with text->text. Reasoning doesn't work text->text.

Re: Gemini 3 Pro: the frontier of vision AI

#122

Post by esafak »

JamesSwift wrote: post #119
danielvaughn wrote: post #57

Earlier quoted context omitted.

I don’t know much about AI, but I have this image test that everything has failed at. You basically just present an image of a maze and ask the LLM to draw a line through the most optimal path. Here’s how Nano Banana fared: https://x.com/danielvaughn/status/1971640520176029704?s=46

I just oneshot it with claude code (opus 4.5) using this prompt. It took about 5 mins and included detecting that it was cheating at first (drew a line around the boundary of the maze instead), so it added guardrails for that: ``` Create a devenv project that does the following: - Read the image at maze.jpg - Write a script that solves the maze in the most optimal way between the mouse and the cheese - Generate a new…

If you allow tool use much simpler models can solve it.

Re: Gemini 3 Pro: the frontier of vision AI

#123

Post by vunderba »

Rover222 wrote: post #64
Workaccount2 wrote: post #48

Well It is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs. In fact GPT5 wrote an edge detection script to see where "golden dog feet" met "bright green grass" to prove to me that there were only 4 legs. The script found 5, and GPT-5 the…

I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned the tail into a weird appendage. Then I asked both Gemini and Grok to count the legs, both kept saying 4. Gemini just refused to consider it was actually wrong. Grok seemed to have an existential crisis when I told it it was wrong, becoming convinced that I ha…

If you want to see something rather amusing - instead of using the LLM aspect of Gemini 3.0 Pro, feed a five-legged dog directly into Nano Banana Pro and give it an editing task that requires an intrinsic understanding of the unusual anatomy.

  Place sneakers on all of its legs.
It'll get this correct a surprising number of times (tested with BFL Flux2 Pro, and NB Pro).

https://imgur.com/a/wXQskhL

Re: Gemini 3 Pro: the frontier of vision AI

#124

Post by runarberg »

CamperBob2 wrote: post #113
runarberg wrote: post #107

Earlier quoted context omitted.

This is the first time I hear the term LLM cognition and I am horrified. LLMs don‘t have cognition. LLMs are a statistical inference machines which predict a given output given some input. There are no mental processes, no sensory information, and certainly no knowledge involved, only statistical reasoning, inference, interpolation, and prediction. Comparing the human mind to an LLM model is like comparing a rubber t…

You'll need to explain the IMO results, then.

Human legs and car tires can both take a human and a car respectively to the finish line of a 200 meter track course, the car tires do so considerably quicker than a pair of human legs. But nobody needs to describe the tire‘s running abilities because of that, nor even compare a tire to a leg. A car tire cannot run, and it is silly to demand an explanation for it.

Re: Gemini 3 Pro: the frontier of vision AI

#125

Post by Zambyte »

irthomasthomas wrote: post #87
Rover222 wrote: post #64

Earlier quoted context omitted.

I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned the tail into a weird appendage. Then I asked both Gemini and Grok to count the legs, both kept saying 4. Gemini just refused to consider it was actually wrong. Grok seemed to have an existential crisis when I told it it was wrong, becoming convinced that I ha…

Isn't this proof that LLMs still don't really generalize beyond their training data?

I wonder how they would behave given a system prompt that asserts "dogs may have more or less than four legs".

Re: Gemini 3 Pro: the frontier of vision AI

#126

Post by MostlyStable »

Going to compare this to our current solution of Amazon's Textract service for analyzing handwritten datasheets. Textract, when extracting tables (which is what we use it for) does not allow for providing any context or information about the tables and what we expect them to contain, but it is really good at correctly recognizing hand written characters. All of my attempts at less specialized, more general models allow me to provide that context, which is helpful in some ways, but fail at the basic part of almost always correctly getting the character.

Hopefully Google pro marries the two together.

Re: Gemini 3 Pro: the frontier of vision AI

#127

Post by runarberg »

Benjammer wrote: post #120
runarberg wrote: post #107

Earlier quoted context omitted.

This is the first time I hear the term LLM cognition and I am horrified. LLMs don‘t have cognition. LLMs are a statistical inference machines which predict a given output given some input. There are no mental processes, no sensory information, and certainly no knowledge involved, only statistical reasoning, inference, interpolation, and prediction. Comparing the human mind to an LLM model is like comparing a rubber t…

>They belong in different categories Categories of _what_, exactly? What word would you use to describe this "kind" of which LLMs and humans are two very different "categories"? I simply chose the word "cognition". I think you're getting hung up on semantics here a bit more than is reasonable.

> Categories of _what_, exactly?

Precisely. At least apples and oranges are both fruits, and it makes sense to compare e.g. the sugar contents of each. But an LLM model and the human brain are as different as the wind and the sunshine. You cannot measure the windspeed of the sun and you cannot measure the UV index of the wind.

Your choice of the words here was rather poor in my opinion. Statistical models do not have cognition any more than the wind has ultra-violet radiation. Cognition is a well studied phenomena, there is a whole field of science dedicated to cognition. And while cognition of animals are often modeled using statistics, statistical models in them selves do not have cognition.

A much better word here would by “abilities”. That is that these tests demonstrate the different abilities of LLM models compared to human abilities (or even the abilities of traditional [specialized] models which often do pass these kinds of tests).

Semantics often do matter, and what worries me is that these statistical models are being anthropomorphized way more then is healthy. People treat them like the crew of the Enterprise treated Data, when in fact they should be treated like the ship‘s computer. And I think this because of a deliberate (and malicious/consumer hostile) marketing campaign from the AI companies.

Re: Gemini 3 Pro: the frontier of vision AI

#128

Post by didgeoridoo »

majormajor wrote: post #96
AIorNot wrote: post #80

Earlier quoted context omitted.

Its not that they aren’t intelligent its that they have been RL’d like crazy to not do that Its rather like as humans we are RL’d like crazy to be grossed out if we view a picture of a handsome man and beautiful woman kissing (after we are told they are brother and sister) - Ie we all have trained biases - that we are told to follow and trained on - human art is about subverting those expectations

Why should I assume that a failure that looks like a model just doing fairly simple pattern matching "this is dog, dogs don't have 5 legs, anything else is irrelevant" vs more sophisticated feature counting of a concrete instance of an entity is RL vs just a prediction failure due to training data not containing a 5-legged dog and an inability to go outside-of-distribution? RL has been used extensively in other areas…

I’m inclined to buy the RL story, since the image gen “deep dream” models of ~10 years ago would produce dogs with TRILLIONS of eyes: https://doorofperception.com/2015/10/google-deep-dream-incep...

Re: Gemini 3 Pro: the frontier of vision AI

#129

Post by phainopepla2 »

Fraterkes wrote: post #115
reducesuffering wrote: post #81

Earlier quoted context omitted.

"AI could never replace the creativity of a human" "Ok, I guess it could wipe out the economic demand for digital art, but it could never do all the autonomous tasks of a project manager" "Ok, I guess it could automate most of that away but there will always be a need for a human engineer to steer it and deal with the nuances of code" "Ok, well it could never automate blue collar work, how is it gonna wrench a pipe i…

The turing test is still a thing. No llm could pass for a person for more than a couple minutes of chatting. That’s a world of difference compared to a decade ago, but I would emphatically not call that “passing the turing test” Also, none of the other things you mentioned have actually happened. Don’t really know why I bother responding to this stuff

> No llm could pass for a person for more than a couple minutes of chatting

I strongly doubt this. If you gave it an appropriate system prompt with instructions and examples on how to speak in a certain way (something different from typical slop, like the way a teenager chats on discord or something), I'm quite sure it could fool the majority of people

Re: Gemini 3 Pro: the frontier of vision AI

#130

Post by biophysboy »

Benjammer wrote: post #116
biophysboy wrote: post #88

Earlier quoted context omitted.

I thought adversarial testing like this was a routine part of software engineering. He's checking to see how flexible it is. Maybe prompting would help, but it would be cool if it was more flexible.

So the idea is what? What's the successful outcome look like for this test, in your mind? What should good software do? Respond and say there are 5 legs? Or question what kind of dog this even is? Or get confused by a nonsensical picture that doesn't quite match the prompt in a confusing way? Should it understand the concept of a dog and be able to tell you that this isn't a real dog?

No, it’s just a test case to demonstrate flexibility when faced with unusual circumstances
Post reply on HN