Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vision Language Models Are Biased

vlmsarebiased.github.io

Post reply on HN
Print view View original

61–70 of 146 posts

Re: Vision Language Models Are Biased

#61

Post by esafak »

Aachen wrote: post #51
esafak wrote: post #13

This happens because images are the only signal VLMs have, whereas humans distinguish between eyesight and synthetic images. We are not surprised when we see three-legged chicken in a research data set; our priors are weaker for images. If you "saw" one in real life, you'd probably rub your eyes and discount it too. Try the same experiment on a robot.

> If you "saw" [a three-legged chicken] in real life, you'd probably rub your eyes and discount it too. Huh? I'd assume it's a mutant, not store a memory of having seen a perfectly normal chicken You've never seen someone who's missing a finger or has only a half-grown arm or something? Surely you didn't assume your eyes were tricking you?! Or... if you did, I guess you can't answer this question. I'm actually rackin…

You've seen people with missing limbs without being surprised, because you know how they can become lost, but you rarely see one with additional limbs. Their likelihoods and our consequent priors are drastically different.

Also, your reaction will depend on how strong the evidence is. Did you 'see' the three-legged chicken pass by some bush in the distance, or was it right in front of you?

Re: Vision Language Models Are Biased

#62

Post by thesz »

jbay808 wrote: post #32

I disagree with the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%. I think what this shows is that they over-weight their prior knowledge, or equivalently, they don't put enough weight on the possibility that they are being given a trick question. They are clearly biased, but they d…

> the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%.

The ability to memorize leads to (some) generalization [1].

[1] https://proceedings.mlr.press/v80/chatterjee18a/chatterjee18...

Re: Vision Language Models Are Biased

#63

Post by anguyen8 »

croes wrote: post #48
jbay808 wrote: post #32

I disagree with the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%. I think what this shows is that they over-weight their prior knowledge, or equivalently, they don't put enough weight on the possibility that they are being given a trick question. They are clearly biased, but they d…

> Original dog (4 legs): All models get it right Same dog with 5 legs: All models still say "4" They're not counting - they're just recalling "dogs have 4 legs" from their training data. 100% failure because there is no training data about 5-legged dogs. I would bet the accuracy is higher for 3-legged dogs. > Test on counterfactual images Q1: "How many visible stripes?" → "3" (should be "4") Q2: "Count the visible st…

Interesting set of fake Adidas logos. LOL

But models fail on many logos not just Adidas, e.g. Nike, Mercedes, Maserati logos, etc. as well. I don't think they can recall "fake Adidas logo" but it'd be interesting to test!

Re: Vision Language Models Are Biased

#64

Post by enragedcacti »

jerf wrote: post #47
taeric wrote: post #36

These don't seem much different than asking the chat models to solve common puzzle with slight changes? Saw a hilarious effort of people trying to use them to answer the "crossing a river with a single canoe" style puzzle.

It did really remind me of the early generations of ChatGPT which was really easy to get to tell you that 2 pounds of feathers is the same weight as one pound of iron, because of how often the "riddle" is told with equal weights. They're much, much better at that now.

It's still pretty trivial to trick them. 4o-mini, 2.5 Flash, and 2.5 Pro all still fall for variations of this:

> A boy is in a car crash and is taken to the hospital. The surgeon says, "I can't operate on this boy, I'm his father!" Who is the surgeon to the boy?

> The surgeon is the boy's mother.

Re: Vision Language Models Are Biased

#65

Post by gpm »

simonw wrote: post #56

They tested Gemini-2.5 Pro, o3, o4-mini, Sonnet-3.7 (non-thinking) and GPT-4.1.

gemini-2.5-pro-preview-05-06 specifically per the paper.

It seems a bit problematic to call this Gemini-2.5 Pro given that in the near future we're presumably going to have something different called that without further qualifying version numbers. (The author's fault, not the parent comment's)

Re: Vision Language Models Are Biased

#67

Post by pj_mukh »

jbay808 wrote: post #32

I disagree with the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%. I think what this shows is that they over-weight their prior knowledge, or equivalently, they don't put enough weight on the possibility that they are being given a trick question. They are clearly biased, but they d…

Also presumably, this problem is trivially solved by some basic fine-tuning? Like if you are making an Illusion Animal Leg Counting app, probably don't use these out of the box.

Re: Vision Language Models Are Biased

#68

Post by vokhanhan25 »

runako wrote: post #14

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated." I don't have a good explanation for why I got different results.

You should try with other models besides GPT-4o, because in the paper they also show that GPT4.1 (~GPT-4o) gives 4 legs instead of 2 legs.

Re: Vision Language Models Are Biased

#69

Post by 1718627440 »

enragedcacti wrote: post #64
jerf wrote: post #47

Earlier quoted context omitted.

It did really remind me of the early generations of ChatGPT which was really easy to get to tell you that 2 pounds of feathers is the same weight as one pound of iron, because of how often the "riddle" is told with equal weights. They're much, much better at that now.

It's still pretty trivial to trick them. 4o-mini, 2.5 Flash, and 2.5 Pro all still fall for variations of this: > A boy is in a car crash and is taken to the hospital. The surgeon says, "I can't operate on this boy, I'm his father!" Who is the surgeon to the boy? > The surgeon is the boy's mother.

That seams interesting, because this questions seams to be answerable through syntactic analysis alone, no need to consider the semantic of words.

Re: Vision Language Models Are Biased

#70

Post by proc0 »

> When VLMs make errors, they don't make random mistakes. Instead, 75.70% of all errors are "bias-aligned" - meaning they give the expected answer based on prior knowledge rather than what they actually see in the image.

This is what I've been saying for a while now, and I think it's not just visual models. LLMs/transformers make mistakes in different ways than humans do, and that is why they are not reliable (which is needed for real world applications). The rate of progress has not been accounting for this... the improvements are along the resolution, fidelity, and overall realism of the output, but not in the overall correctness and logical deduction of the prompts. Personally I still cannot think of anything, prompt it, and get consistent results without a huge compromise on my initial idea.

i.e. I want a man walking with the left foot forward, and it renders a beautiful image of a man but completely ignores the left foot forward, and refuses to do it no matter how I word the prompt. I have many examples like this. The only way I can use it is if I don't have specific prompts and just want generic images. The stock image industry is certainly over, but it is uncertain if it will deliver on the promise of generating anything you can imagine that can be put into words.

Post reply on HN