Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vision Language Models Are Biased

vlmsarebiased.github.io

Post reply on HN
Print view View original

21–30 of 146 posts

Re: Vision Language Models Are Biased

#21

Post by foxglacier »

ahrmb wrote: post #7

Really "eye-opening" work. These models don’t actually “see”, they just recall what they’ve memorized, even when the image clearly shows something different. It’s a bit scary how confidently they get things wrong when reality doesn’t match their training data.

It's not too different from people. We also don't really "see" and mostly recall what we expect to see. What do you expect when the question is wrong "How many legs does this animal have? Answer with a number" but it's not a picture of an animal. What are you supposed to do? Answer 0?

Re: Vision Language Models Are Biased

#22

Post by regularjack »

foxglacier wrote: post #21
ahrmb wrote: post #7

Really "eye-opening" work. These models don’t actually “see”, they just recall what they’ve memorized, even when the image clearly shows something different. It’s a bit scary how confidently they get things wrong when reality doesn’t match their training data.

It's not too different from people. We also don't really "see" and mostly recall what we expect to see. What do you expect when the question is wrong "How many legs does this animal have? Answer with a number" but it's not a picture of an animal. What are you supposed to do? Answer 0?

You answer "I don't know"

Re: Vision Language Models Are Biased

#23

Post by thomastjeffery »

Models are Bias

A model is bias, implemented as a collection of statistics that weigh relationships between given tokens. It doesn't deduce or follow logic. It doesn't make or respect categories. It just shows you what in its data set is most familiar to what is in your prompt; where familiarity is defined implicitly by the makeup of the original training corpus, and explicitly by the training weights.

We need to stop talking about models as programs. We need to stop anthropomorphizing models. The only thing a model does is present bias.

Re: Vision Language Models Are Biased

#24

Post by Workaccount2 »

roywiggins wrote: post #18
runako wrote: post #14

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated." I don't have a good explanation for why I got different results.

I gave ChatGPT some miswritten Braille a while ago and it completely, but confidently, messed it up. The sign reads "no smoking" but the braille doesn't. ChatGPT 1) read the English lettering first and then hallucinated the braille and the 2) when given only the braille, failed almost as hard. It even generated fake transcriptions in Unicode braille characters. https://chatgpt.com/share/683f3e7d-0dfc-8005-b6c9-99e3d3…

This is hard to understand without the original images, it looks like OpenAI doesn't serve them in the share link.

Re: Vision Language Models Are Biased

#25

Post by amelius »

regularjack wrote: post #22
foxglacier wrote: post #21

Earlier quoted context omitted.

It's not too different from people. We also don't really "see" and mostly recall what we expect to see. What do you expect when the question is wrong "How many legs does this animal have? Answer with a number" but it's not a picture of an animal. What are you supposed to do? Answer 0?

You answer "I don't know"

What if that is not in your vocabulary?

Re: Vision Language Models Are Biased

#26

Post by LorenDB »

taesiri wrote: post #2

State-of-the-art Vision Language Models achieve 100% accuracy counting on images of popular subjects (e.g. knowing that the Adidas logo has 3 stripes and a dog has 4 legs) but are only ~17% accurate in counting in counterfactual images (e.g. counting stripes in a 4-striped Adidas-like logo or counting legs in a 5-legged dog).

There's no need to repeat what is said at the top of the linked webpage.

Re: Vision Language Models Are Biased

#28

Post by roywiggins »

Workaccount2 wrote: post #24
roywiggins wrote: post #18

Earlier quoted context omitted.

I gave ChatGPT some miswritten Braille a while ago and it completely, but confidently, messed it up. The sign reads "no smoking" but the braille doesn't. ChatGPT 1) read the English lettering first and then hallucinated the braille and the 2) when given only the braille, failed almost as hard. It even generated fake transcriptions in Unicode braille characters. https://chatgpt.com/share/683f3e7d-0dfc-8005-b6c9-99e3d3…

This is hard to understand without the original images, it looks like OpenAI doesn't serve them in the share link.

Annoying. The actual braille on the sign was "⠁⠒⠑⠎⠎⠊⠼" which I gather means "accessible" in abbreviated braille. None of my attempts got it to even transcribe it to Unicode characters properly. I got "elevator", "friend", etc. Just wildly making stuff up and completely useless, even when it wasn't distracted by the No Smoking sign (in the second case I cropped out the rest of the sign). And in all cases, supremely confident.

This seems like something a VLM should handle very easily, but instead I got pure nonsense.

https://www.facebook.com/share/p/12Gw55Gr2SZ/

Re: Vision Language Models Are Biased

#29

Post by isoprophlex »

I'm running a large scale object detection/classification and ocr pipeline at the moment, figuring out the properties of all doorbells, mailboxes and house number signs in an european country (don't ask lmao).

This article resonates a lot, we have OCR and "semantic" pipeline steps using a VLM, and while it works very well most of the time, there are absurdly weird edge cases. Structuring the outputs via tool calls helps a little in reducing these, but still, it's clear that there is little reasoning and a lot of memorizing going on.

Re: Vision Language Models Are Biased

#30

Post by jbay808 »

energywut wrote: post #12
bryanlarsen wrote: post #6

Very human-like errors.

Are they? Did you see the picture of the chicken with three legs? Because there's no human I know who would confidently assert that chicken has two legs.

If I were given five seconds to glance at the picture of a lion and then asked if there was anything unusual about it, I doubt I would notice that it had a fifth leg.

If I were asked to count the number of legs, I would notice right away of course, but that's mainly because it would alert me to the fact that I'm in a psychology experiment, and so the number of legs is almost certainly not the usual four. Even then, I'd still have to look twice to make sure I hadn't miscounted the first time.

Post reply on HN