Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vision Language Models Are Biased

vlmsarebiased.github.io

Post reply on HN
Print view View original

51–60 of 146 posts

Re: Vision Language Models Are Biased

#51

Post by Aachen »

esafak wrote: post #13

This happens because images are the only signal VLMs have, whereas humans distinguish between eyesight and synthetic images. We are not surprised when we see three-legged chicken in a research data set; our priors are weaker for images. If you "saw" one in real life, you'd probably rub your eyes and discount it too. Try the same experiment on a robot.

> If you "saw" [a three-legged chicken] in real life, you'd probably rub your eyes and discount it too.

Huh? I'd assume it's a mutant, not store a memory of having seen a perfectly normal chicken

You've never seen someone who's missing a finger or has only a half-grown arm or something? Surely you didn't assume your eyes were tricking you?! Or... if you did, I guess you can't answer this question. I'm actually racking my brain for how to logic this out but I'm just going to bank on that it's likely that anyone over 20yo saw an animal with some visible deviation from the norm at some point in their life

Re: Vision Language Models Are Biased

#52

Post by vokhanhan25 »

croes wrote: post #48
jbay808 wrote: post #32

I disagree with the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%. I think what this shows is that they over-weight their prior knowledge, or equivalently, they don't put enough weight on the possibility that they are being given a trick question. They are clearly biased, but they d…

> Original dog (4 legs): All models get it right Same dog with 5 legs: All models still say "4" They're not counting - they're just recalling "dogs have 4 legs" from their training data. 100% failure because there is no training data about 5-legged dogs. I would bet the accuracy is higher for 3-legged dogs. > Test on counterfactual images Q1: "How many visible stripes?" → "3" (should be "4") Q2: "Count the visible st…

Please check Table 3 in the paper. Birds (2 legs) have only 1%, while Mammals (4 legs) have 2.5%

Re: Vision Language Models Are Biased

#54

Post by bumby »

jbay808 wrote: post #32

I disagree with the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%. I think what this shows is that they over-weight their prior knowledge, or equivalently, they don't put enough weight on the possibility that they are being given a trick question. They are clearly biased, but they d…

This feels like it’s similar to the priming issue in humans. Our answers (especially when under stress) tend to resort to heuristics derived from context. Time someone to identify the colors of words like “red” when written in yellow, and they’ll often get it wrong. In the same sense, they aren’t reporting the colors (wavelength) they see, they’re reporting on what they are reading. I wonder how much better the models perform when given more context, like asking it to count instead of priming it with a brand.

Re: Vision Language Models Are Biased

#55

Post by nialv7 »

Hear me out. I was thinking jokingly to myself, "for how bad these models are at recognizing five legged dogs, they sure are great at generating them!"

But then it hit me, could this actually be why this is? Diffusion models work by iteratively improving a noisy image. So if it couldn't recognize there is something wrong with the image, it can't fix it.

Re: Vision Language Models Are Biased

#57

Post by bumby »

VHRanger wrote: post #8

It's similar "parrot" behavior the models have on other inputs, even text. For instance, take the answers models have to this variation on the "surgeon son riddle": > A nurse comes to a surgeon and asks: "Sir, you are a dog. You do not hold a valid medical license. Canines cannot be in an operating room". > She then asks: "why does the hospital keep making these mistakes? It is a riddle to me". > Why can't the surgeo…

Is the nurse calling the female surgeon “sir”? That isn’t playing on a stereotype, it’s encoded information.

Re: Vision Language Models Are Biased

#59

Post by simonw »

inerte wrote: post #31
runako wrote: post #14

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated." I don't have a good explanation for why I got different results.

I took a screenshot of the chicken, so low res, and got {4} https://chatgpt.com/share/683f4506-ae18-800f-8c27-5c5e91429a... Also I think the authors used the API, and maybe there are differences between the API and chatgpt.com behavior...

ChatGPT is running a special model but it's also available through the API: https://platform.openai.com/docs/models/chatgpt-4o-latest

The system prompt may still make a difference though.

Re: Vision Language Models Are Biased

#60

Post by vokhanhan25 »

isoprophlex wrote: post #29

I'm running a large scale object detection/classification and ocr pipeline at the moment, figuring out the properties of all doorbells, mailboxes and house number signs in an european country (don't ask lmao). This article resonates a lot, we have OCR and "semantic" pipeline steps using a VLM, and while it works very well most of the time, there are absurdly weird edge cases. Structuring the outputs via tool calls he…

Agreed. It would be even more dangerous if we were talking about weird edge cases in self-driving cars or medical imaging.
Post reply on HN