Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Pro: the frontier of vision AI

blog.google

Post reply on HN
Print view View original

191–200 of 309 posts

Re: Gemini 3 Pro: the frontier of vision AI

#191

Post by Lamprey »

vunderba wrote: post #123
Rover222 wrote: post #64

Earlier quoted context omitted.

I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned the tail into a weird appendage. Then I asked both Gemini and Grok to count the legs, both kept saying 4. Gemini just refused to consider it was actually wrong. Grok seemed to have an existential crisis when I told it it was wrong, becoming convinced that I ha…

If you want to see something rather amusing - instead of using the LLM aspect of Gemini 3.0 Pro, feed a five-legged dog directly into Nano Banana Pro and give it an editing task that requires an intrinsic understanding of the unusual anatomy . Place sneakers on all of its legs. It'll get this correct a surprising number of times (tested with BFL Flux2 Pro, and NB Pro). https://imgur.com/a/wXQskhL

Does this still work if you give it a pre-existing many-legged animal image, instead of first prompting it to add an extra leg and then prompting it to put the sneakers on all the legs?

I'm wondering if it may only expect the additional leg because you literally just told it to add said additional leg. It would just need to remember your previous instruction and its previous action, rather than to correctly identify the number of legs directly from the image.

I'll also note that photos of dogs with shoes on is definitely something it has been trained on, albeit presumably more often dog booties than human sneakers.

Can you make it place the sneakers incorrectly-on-purpose? "Place the sneakers on all the dog's knees?"

Re: Gemini 3 Pro: the frontier of vision AI

#192

Post by mkl »

RestartKernel wrote: post #159
macNchz wrote: post #144

Earlier quoted context omitted.

An interesting test in this vein that I read about in a comment on here is generating a 13 hour clock—I tried just about every prompting trick and clever strategy I could come up with across many image models with no success. I think there's so much training data of 12 hour clocks that just clobbers the instructions entirely. It'll make a regular clock that skips from 11 to 13, or a regular clock with a plaque saying…

Right you are. It can do 26 hours just fine, but appears completely incapable when the layout would be too close to a normal clock. https://gemini.google.com/share/b3b68deaa6e6 I thought giving it a setting would help, but just skip that first response to see what I mean.

That's a 24 hour clock that skips some numbers and puts other numbers out of order.

Re: Gemini 3 Pro: the frontier of vision AI

#193

Post by Lamprey »

didgeoridoo wrote: post #128
majormajor wrote: post #96

Earlier quoted context omitted.

Why should I assume that a failure that looks like a model just doing fairly simple pattern matching "this is dog, dogs don't have 5 legs, anything else is irrelevant" vs more sophisticated feature counting of a concrete instance of an entity is RL vs just a prediction failure due to training data not containing a 5-legged dog and an inability to go outside-of-distribution? RL has been used extensively in other areas…

I’m inclined to buy the RL story, since the image gen “deep dream” models of ~10 years ago would produce dogs with TRILLIONS of eyes: https://doorofperception.com/2015/10/google-deep-dream-incep...

That's apples to oranges; your link says they made it exaggerate features on purpose.

"The researchers feed a picture into the artificial neural network, asking it to recognise a feature of it, and modify the picture to emphasise the feature it recognises. That modified picture is then fed back into the network, which is again tasked to recognise features and emphasise them, and so on. Eventually, the feedback loop modifies the picture beyond all recognition."

Re: Gemini 3 Pro: the frontier of vision AI

#194

Post by tensegrist »

vunderba wrote: post #123
Rover222 wrote: post #64

Earlier quoted context omitted.

I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned the tail into a weird appendage. Then I asked both Gemini and Grok to count the legs, both kept saying 4. Gemini just refused to consider it was actually wrong. Grok seemed to have an existential crisis when I told it it was wrong, becoming convinced that I ha…

If you want to see something rather amusing - instead of using the LLM aspect of Gemini 3.0 Pro, feed a five-legged dog directly into Nano Banana Pro and give it an editing task that requires an intrinsic understanding of the unusual anatomy . Place sneakers on all of its legs. It'll get this correct a surprising number of times (tested with BFL Flux2 Pro, and NB Pro). https://imgur.com/a/wXQskhL

i imagine the real answer is that the edits are local because that's how diffusion works; it's not like it's turning the input into "five-legged dog" and then generating a five-legged dog in shoes from scratch

Re: Gemini 3 Pro: the frontier of vision AI

#195

Post by vunderba »

Lamprey wrote: post #191
vunderba wrote: post #123

Earlier quoted context omitted.

If you want to see something rather amusing - instead of using the LLM aspect of Gemini 3.0 Pro, feed a five-legged dog directly into Nano Banana Pro and give it an editing task that requires an intrinsic understanding of the unusual anatomy . Place sneakers on all of its legs. It'll get this correct a surprising number of times (tested with BFL Flux2 Pro, and NB Pro). https://imgur.com/a/wXQskhL

Does this still work if you give it a pre-existing many-legged animal image, instead of first prompting it to add an extra leg and then prompting it to put the sneakers on all the legs? I'm wondering if it may only expect the additional leg because you literally just told it to add said additional leg. It would just need to remember your previous instruction and its previous action, rather than to correctly identify…

My example was unclear. Each of those images on Imgur was generated using independent API calls which means there was no "rolling context/memory".

In other words:

1. Took a personal image of my dog Lily

2. Had NB Pro add a fifth leg using the Gemini API

3. Downloaded image

4. Sent image to BFL Flux2 Pro via the BFL API with the prompt "Place sneakers on all the legs of this animal".

5. Sent image to NB Pro via Gemini API with the prompt "Place sneakers on all the legs of this animal".

So not only was there zero "continual context", it was two entirely different models as well to cover my bases.

EDIT: Added images to the Imgur for the following prompts:

- Place red Dixie solo cups on the ends of every foot on the animal

- Draw a red circle around all the feet on the animal

Re: Gemini 3 Pro: the frontier of vision AI

#196

Post by energy123 »

runarberg wrote: post #182
energy123 wrote: post #175

Earlier quoted context omitted.

I think we are too quick to discount the possibility that this flaw is slightly intentional, in the sense that the optimization has a tight budget to work with (equivalent of ~3000 tokens) so why would it waste capacity on this when it could improve capabilities around reading small text in obscured images? Sort of like humans have all these rules of thumbs that backfire in all these ways but that's the energy effici…

Even so, that doesn’t take away from my point. Traditional specialized models can do these things already, for much cheaper and without expensive optimization. What traditional models cannot do is the toy aspect of LLM, and that is the only usecase I see for this technology going forward. Lets say you are right and these things will be optimized, and in, say, 5 years, most models from the big players will be able do…

> Why would people use LLMs for these when a traditional specialized model can do it for much cheaper?

This is not too different from where I see things going. I don't think a monolithic LLM that does everything perfectly is where we'll go. An LLM in a finite-compute universe is never going to be better at weather forecasting than GraphCast. The LLM will have a finite compute budget, and it should prioritize general reasoning, and be capable of calling tools like GraphCast to extend its intelligence into the necessary verticals for solving a problem.

I don't know exactly what that balance will look like however, and the lines between specialist application knowledge and general intelligence is pretty blurred, and what the API boundaries (if any) should be are unclear to me. There's a phenomenon where capabilities in one vertical do help with general reasoning to an extent, so it's not a completely zero-sum tradeoff between specialist expertise and generalist abilities, which makes it difficult to know what to expect.

Re: Gemini 3 Pro: the frontier of vision AI

#197

Post by Frannky »

It's a good model. I worry that they will be able to win the game by offering the best service for free, thanks to selling users' data—kind of like search, email, etc. It's sad. Not that the alternatives are better... You either trust synchopathic ChatGPT backed by Scama, go with woke Claude (they once banned my account for asking how some news was trying to influence me), Grok that feels like a 20-year-old sure about stuff that don't work, and Chinese models that are agenda-aligned...

Re: Gemini 3 Pro: the frontier of vision AI

#198

Post by TheAceOfHearts »

genrader wrote: post #170
TheAceOfHearts wrote: post #56

Since I think it's interesting to highlight the jagged intelligence, I have a simple word search puzzle [0] that Nano Banana Pro stills struggles to solve correctly. Gemini 3 Pro with Code Execution is able to one-shot the problem and find the positions of each word (this is super impressive! one year ago it wasn't possible), but Nano Banana Pro fails to highlight the words correctly. Here's the output from two tests…

If you're using for instance the Gemini web app there may be a preference in the system prompt to immediately favor the fact that you said to create an image when in fact it may have been better to initially start with a regular chat prompt, making sure you're on Gemini 3 Pro thinking, and then give it exactly what you usually would. You can tell it that after it has an answer to the question then to create an image…

I just used Nano Banana Pro from LMArena, but if you have access to a paid account I'd love to see you try it out! I just gave it the puzzle image as an input along with the prompt: "Please solve this word search puzzle".

For generating the prompt which included the word positions I had Gemini 3 Pro do that using the following prompt: "Please try to solve this word search puzzle. Give me the position of each word in the grid. Then generate a prompt which I can pass to Nano Banana Pro, which I will pass along with the same input image to see if Nano Banana Pro is able to properly highlight all the words if given their correct position."

Re: Gemini 3 Pro: the frontier of vision AI

#199

Post by HardCodedBias »

AIorNot wrote: post #80
Rover222 wrote: post #64

Earlier quoted context omitted.

I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned the tail into a weird appendage. Then I asked both Gemini and Grok to count the legs, both kept saying 4. Gemini just refused to consider it was actually wrong. Grok seemed to have an existential crisis when I told it it was wrong, becoming convinced that I ha…

Its not that they aren’t intelligent its that they have been RL’d like crazy to not do that Its rather like as humans we are RL’d like crazy to be grossed out if we view a picture of a handsome man and beautiful woman kissing (after we are told they are brother and sister) - Ie we all have trained biases - that we are told to follow and trained on - human art is about subverting those expectations

"There are four lights"

And the AI has been RLed for tens of thousands of years not just a few days.

Re: Gemini 3 Pro: the frontier of vision AI

#200

Post by knollimar »

ftcHn wrote: post #162
knollimar wrote: post #86

Earlier quoted context omitted.

I've been using pyrevit inside revit so I just threw a basic loop in there. There's already a building model and the coworkers are just placing and wiring outlets, switches, etc. The harness wasn't impressive enough to share (alos contains vibe coded UI since I didn't want to learn XAML stuff on a friday night). Nothing fancy; I'm not very skilled (I work in construction) I gave it some custom methods it could call,…

I'm going to try to get it to generate extrusions in Revit based on images of floor plans. I've tried doing this in bunch of models without success so far.

You might want to give it some guidance based on edge centers? It'll have a hard time thinking of wall thickness and have it draw points if you're trying to copy floor plans.

for clarity now that I'm rereading: it understands vectors a lot better than areas. Encoding it like that seems to work better for me.

Post reply on HN