Vision Language Models Are Biased • Hacker News Board

Re: Vision Language Models Are Biased

#71

Post by miguel_martin » Tue, Jun 03, 2025, 9:02 PM UTC

gamerDude wrote: post #19

Hypothetically, could this be fixed by changing the input method. For instance, I just quickly looked up how humans process imagery. "the primary visual cortex, located at the back of the brain, receives the visual signals and processes basic visual features like edges, lines, and orientations." So, potentially if we did a pre-processing step to get more features out beforehand we would see different results in the o…

There are enough features fed into a VLM to solve the task.

The way to fix this is simpler: ensure counter-factuals are present in the training data, then the VLM will learn not to be dependent on its language priors/knowledge.

Re: Vision Language Models Are Biased

#72

Post by ramoz » Tue, Jun 03, 2025, 9:02 PM UTC

zehaeva wrote: post #40

ramoz wrote: post #34

Earlier quoted context omitted.

This is interesting actually. And reminds me of something vaguely - a book or something that describes how human attention and the things we see are highly optimized by evolution. We often miss a lot of details in reality due to this.

If it were a Fiction novel then might I suggest Blindsight by Peter Watts?

not fiction. Maybe like a System 1 vs System 2 thing from Thinking, Fast and Slow by Kahneman.

ChatGPT mentioned The Case Against Reality but I never read that, the idea was similar.

Re: Vision Language Models Are Biased

#73

Post by anguyen8 » Tue, Jun 03, 2025, 9:02 PM UTC

inerte wrote: post #31

runako wrote: post #14

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated." I don't have a good explanation for why I got different results.

I took a screenshot of the chicken, so low res, and got {4} https://chatgpt.com/share/683f4506-ae18-800f-8c27-5c5e91429a... Also I think the authors used the API, and maybe there are differences between the API and chatgpt.com behavior...

https://imgur.com/cO7eFNt

o3 Chat is also similarly wrong, saying {4}.

Re: Vision Language Models Are Biased

#74

Post by rafram » Tue, Jun 03, 2025, 9:07 PM UTC

This won't be a surprise to anyone who's tried using a VLM on text. When it can't read a word (or an entire passage), it just outputs what it expects to see. That's far worse than a traditional OCR failure because it's often what you expect to see, too, so it's quite hard to catch in a manual review.

Re: Vision Language Models Are Biased

#75

Post by impossiblefork » Tue, Jun 03, 2025, 9:20 PM UTC

taesiri wrote: post #5

shenkha wrote: post #3

fun findings related to memorization of AI models. It simply means LLMs/VLLMs do not know how to predict generally but memorizing instead. A new perspective on adversarial attack methods.

for overly represented concepts, like popular brands, it seems that the model “ignores” the details once it detects that the overall shapes or patterns are similar. Opening up the vision encoders to find out how these images cluster in the embedding space should provide better insights.

Yes, and this can probably be solved by methods for fairness.

I used to believe that fairness research could be ignored, that it was all rubbish, but they at least try to do something about things like unbalanced datasets etc. I'm still not sure I totally believe in it though.

Re: Vision Language Models Are Biased

#76

Post by gowld » Tue, Jun 03, 2025, 9:24 PM UTC

vunderba wrote: post #39

foxglacier wrote: post #21

Earlier quoted context omitted.

It's not too different from people. We also don't really "see" and mostly recall what we expect to see. What do you expect when the question is wrong "How many legs does this animal have? Answer with a number" but it's not a picture of an animal. What are you supposed to do? Answer 0?

That wasn't one of the questions - any reasonable person would have classified that chicken as an animal, albeit a mutant one. I would also hardly count many of these questions as "tricks" either. Take the chess example. A lot of my friends and myself have been playing chess since we were young children and we all know that a fully populated chess board has 32 pieces (heavily weighted in our internal training data),…

Don't be too literal.

Imagine walking to a room an seeing someone grab a handful of chess pieces off of a set-up board, and proceed to fill bags with 4 pieces each. As they fill the 8th bag, they notice only 3 pieces are left. Are you confident that you would respond "I saw the board only had 31 pieces on it when you started", or might you reply "perhaos you dropped a piece on the floor"?

Re: Vision Language Models Are Biased

#77

Post by throwaway7783 » Tue, Jun 03, 2025, 9:30 PM UTC

Unless the training set was explicitly biased in a specific way, this is basically saying that "the world is biased"

Re: Vision Language Models Are Biased

#78

Post by enragedcacti » Tue, Jun 03, 2025, 9:33 PM UTC

bryanlarsen wrote: post #16

energywut wrote: post #12

Earlier quoted context omitted.

Are they? Did you see the picture of the chicken with three legs? Because there's no human I know who would confidently assert that chicken has two legs.

Throw 1000 pictures of chickens at a human, ask how many legs each chicken has. If 999 of them have two, I bet you'll get two as an answer back for the 1000th one no matter how obvious.

Humans do things a lot harder than that every day in the form of QA in factories. Do they sometimes make mistakes from the repetition or boredom? Sure. Is that at all comparable to the failures in the paper? No.

Re: Vision Language Models Are Biased

#79

Post by conception » Tue, Jun 03, 2025, 9:40 PM UTC

proc0 wrote: post #70

> When VLMs make errors, they don't make random mistakes. Instead, 75.70% of all errors are "bias-aligned" - meaning they give the expected answer based on prior knowledge rather than what they actually see in the image. This is what I've been saying for a while now, and I think it's not just visual models. LLMs/transformers make mistakes in different ways than humans do, and that is why they are not reliable (which…

https://chatgpt.com/s/m_683f6b9dbb188191b7d735b247d894df

I think this used to be the case in the way that you used to not be able to draw a picture of a bowl of Ramen without chopsticks, but I think the latest models account for this and are much better.

Re: Vision Language Models Are Biased

#80

Post by enragedcacti » Tue, Jun 03, 2025, 9:47 PM UTC

foxglacier wrote: post #21

ahrmb wrote: post #7

Really "eye-opening" work. These models don’t actually “see”, they just recall what they’ve memorized, even when the image clearly shows something different. It’s a bit scary how confidently they get things wrong when reality doesn’t match their training data.

It's not too different from people. We also don't really "see" and mostly recall what we expect to see. What do you expect when the question is wrong "How many legs does this animal have? Answer with a number" but it's not a picture of an animal. What are you supposed to do? Answer 0?

Its true that our brains take lots of shortcuts when processing visual information but they don't necessarily parallel the shortcuts VLMs take. Humans are often very good at identifying anomalous instances of things they've seen thousands of times. No one has to tell you to look closely when you look at your partner in a mirror, you'll recognize it as 'off' immediately. Same for uncanny CGI of all types of things. If we were as sloppy as these models then VFX would be a hell of a lot easier.

Ironically I think a lot of people in this thread are remembering things they learned about the faultiness of humans' visual memory and applying it to visual processing.