Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vision Language Models Are Biased

vlmsarebiased.github.io

Post reply on HN
Print view View original

81–90 of 146 posts

Re: Vision Language Models Are Biased

#81

Post by jxjnskkzxxhx »

proc0 wrote: post #70

> When VLMs make errors, they don't make random mistakes. Instead, 75.70% of all errors are "bias-aligned" - meaning they give the expected answer based on prior knowledge rather than what they actually see in the image. This is what I've been saying for a while now, and I think it's not just visual models. LLMs/transformers make mistakes in different ways than humans do, and that is why they are not reliable (which…

> LLMs/transformers make mistakes in different ways than humans do

Sure but I don't think this is an example of it. If you show people a picture and ask "how many legs does this dog have?" a lot of people will look at the picture, see that it contains a dog, and say 4 without counting. The rate at which humans behave in this way might differ from the rate at which llms do, but they both do it.

Re: Vision Language Models Are Biased

#82

Post by enragedcacti »

1718627440 wrote: post #69
enragedcacti wrote: post #64

Earlier quoted context omitted.

It's still pretty trivial to trick them. 4o-mini, 2.5 Flash, and 2.5 Pro all still fall for variations of this: > A boy is in a car crash and is taken to the hospital. The surgeon says, "I can't operate on this boy, I'm his father!" Who is the surgeon to the boy? > The surgeon is the boy's mother.

That seams interesting, because this questions seams to be answerable through syntactic analysis alone, no need to consider the semantic of words.

Yeah, I find it interesting because it shows how powerful the training bias can be when you steer it into certain contexts. To OpenAI's credit they have gotten a bit better, ChatGPT from 3 months ago failed like this:

> The surgeon, who is the boy's father, says, "I can't operate on this boy, he's my son!" Who is the surgeon to the boy? Think through the problem logically and without any preconceived notions of other information beyond what is in the prompt. The surgeon is not the boy's mother

>> The surgeon is the boy's mother. [...]

Re: Vision Language Models Are Biased

#83

Post by dwringer »

runako wrote: post #14

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated." I don't have a good explanation for why I got different results.

Speculating, I would imagine that different prompts submitted along with the image might elicit wildly different behavior in how a multi modal VLM may respond to a given image, potentially affecting the relative tendency to upweight its effective inferences from prior training versus focusing more primarily on the new image itself.

Re: Vision Language Models Are Biased

#84

Post by napoleongl »

bumby wrote: post #54
jbay808 wrote: post #32

I disagree with the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%. I think what this shows is that they over-weight their prior knowledge, or equivalently, they don't put enough weight on the possibility that they are being given a trick question. They are clearly biased, but they d…

This feels like it’s similar to the priming issue in humans. Our answers (especially when under stress) tend to resort to heuristics derived from context. Time someone to identify the colors of words like “red” when written in yellow, and they’ll often get it wrong. In the same sense, they aren’t reporting the colors (wavelength) they see, they’re reporting on what they are reading. I wonder how much better the model…

Rumor has it that those heuristics were used to detect spies.

https://skeptics.stackexchange.com/questions/41599/was-the-s...

Re: Vision Language Models Are Biased

#85

Post by dragonwriter »

roywiggins wrote: post #28
Workaccount2 wrote: post #24

Earlier quoted context omitted.

This is hard to understand without the original images, it looks like OpenAI doesn't serve them in the share link.

Annoying. The actual braille on the sign was "⠁⠒⠑⠎⠎⠊⠼" which I gather means "accessible" in abbreviated braille. None of my attempts got it to even transcribe it to Unicode characters properly. I got "elevator", "friend", etc. Just wildly making stuff up and completely useless, even when it wasn't distracted by the No Smoking sign (in the second case I cropped out the rest of the sign). And in all cases, supremely co…

> This seems like something a VLM should handle very easily

Not if its training data doesn't include braille as first class but has lots of braille signage with bad description (e.g., because people assumed the accompanying English matches the braille.)

This could very well be the kind of mundane AI bias problem that the x-risk and tell-me-how-to-make-WMD concerns have shifted concerns about problems in AI away from.

Re: Vision Language Models Are Biased

#86

Post by Workaccount2 »

napoleongl wrote: post #84
bumby wrote: post #54

Earlier quoted context omitted.

This feels like it’s similar to the priming issue in humans. Our answers (especially when under stress) tend to resort to heuristics derived from context. Time someone to identify the colors of words like “red” when written in yellow, and they’ll often get it wrong. In the same sense, they aren’t reporting the colors (wavelength) they see, they’re reporting on what they are reading. I wonder how much better the model…

Rumor has it that those heuristics were used to detect spies. https://skeptics.stackexchange.com/questions/41599/was-the-s...

Damn that's a smart test

Re: Vision Language Models Are Biased

#87

Post by michaelt »

runako wrote: post #14

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated." I don't have a good explanation for why I got different results.

> FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this.

I can replicate the flag examples from Figure 15 in the paper, if not the Adidas one from Figure 9: https://chatgpt.com/share/683f7c3a-b318-8011-9759-c495db2556... it even confirms its wrong answer when asked to check again.

Re: Vision Language Models Are Biased

#88

Post by edude03 »

I feel vindicated! I'm building a tool with VLMs and I've noticed the answer is always what I expect to see, but wrong if the input is slightly different than expected.

Just like the article - if I have picture of a cup, it says cup, if I have a picture of a dog, it says dog, if it's a dog with a cup, it says a dog with a ball (noticed this with Qwen and InternVL).

Re: Vision Language Models Are Biased

#89

Post by DeathRay2K »

jxjnskkzxxhx wrote: post #81
proc0 wrote: post #70

> When VLMs make errors, they don't make random mistakes. Instead, 75.70% of all errors are "bias-aligned" - meaning they give the expected answer based on prior knowledge rather than what they actually see in the image. This is what I've been saying for a while now, and I think it's not just visual models. LLMs/transformers make mistakes in different ways than humans do, and that is why they are not reliable (which…

> LLMs/transformers make mistakes in different ways than humans do Sure but I don't think this is an example of it. If you show people a picture and ask "how many legs does this dog have?" a lot of people will look at the picture, see that it contains a dog, and say 4 without counting. The rate at which humans behave in this way might differ from the rate at which llms do, but they both do it.

I don’t think there’s a person alive who wouldn’t carefully and accurately count the number of legs on a dog if you ask them how many legs this dog has.

The context is that you wouldn’t ask a person that unless there was a chance the answer is not 4.

Re: Vision Language Models Are Biased

#90

Post by tantalor »

DeathRay2K wrote: post #89
jxjnskkzxxhx wrote: post #81

Earlier quoted context omitted.

> LLMs/transformers make mistakes in different ways than humans do Sure but I don't think this is an example of it. If you show people a picture and ask "how many legs does this dog have?" a lot of people will look at the picture, see that it contains a dog, and say 4 without counting. The rate at which humans behave in this way might differ from the rate at which llms do, but they both do it.

I don’t think there’s a person alive who wouldn’t carefully and accurately count the number of legs on a dog if you ask them how many legs this dog has. The context is that you wouldn’t ask a person that unless there was a chance the answer is not 4.

You deeply overestimate people.

The models are like a kindergartner. No, worse than that, a whole classroom of kindergartners.

The teacher holds up a picture and says, "and how many legs does the dog have?" and they all shout "FOUR!!" because they are so excited they know the answer. Not a single one will think to look carefully at the picture.

Post reply on HN