Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vision language models are blind

vlmsareblind.github.io

Post reply on HN
Print view View original

191–200 of 202 posts

Re: Vision language models are blind

#191

Post by ben_w »

ClumsyPilot wrote: post #148
godelski wrote: post #87

Earlier quoted context omitted.

> Did they try to probe that hypothesis at all? I think this is a communication issue and you're being a bit myopic in your interpretation. It is clearly an analogy meant for communication and is not an actual hypothesis. Sure, they could have used a better analogy and they could have done other tests, but the paper still counters quite common claims (from researchers) about VLMs. > I could (well actually I can't) sh…

> Second, so what if a test was designed to trick up a model? Shouldn't we be determining when and where models fail? Is that not a critical question in understanding how to use them properly? People are rushing to build this AI into all kinds of products, and they actively don’t want to know where the problems are. The real world outside is designed to trip up the model. Strange things happen all the time. Because s…

> Because software developers have no governing body, no oaths of ethics and no spine someone will end up dead in a ditch from malfunctioning AI.

The conclusion and the premise are both true, but not the causality. On AI, the Overton window is mostly filled with people going "this could be very bad if we get it wrong".

Unfortunately, there's enough people who think "unless I do it first" (Musk, IMO) or "it can't possibly be harmful" (LeCun) that it will indeed kill more people than it already has.

The number who are already (and literally) "dead in a ditch" is definitely above zero if you include all the things that used to be AI when I was a kid e.g. "route finding": https://www.cbsnews.com/news/google-sued-negligence-maps-dri...

Re: Vision language models are blind

#192

Post by joefourier »

RodgerTheGreat wrote: post #5
rezaghanbari1 wrote: post #3

Some of these samples are shocking. How do these models answer chart-based questions, I mean when they can't even count the intersections between two lines?

Same way they answer any question: piece together a statistically probable sequence of words to follow the prompt. All they know about an image is a handful of words a classifier might choose to describe it. If those words have nothing to do with the question being asked, they can't nudge the model in the general direction of a correct answer, so it's a crapshoot- even moreso than usual.

That’s not at all how multi-modal LLMs work - their visual input is not words generated by a classifier. Instead the image is divided into patches and tokenised by a visual encoder (essentially, it is compressed), and then fed directly as a sequence to the model.

Re: Vision language models are blind

#193

Post by mr_toad »

slashdave wrote: post #186
jordan_bonecut wrote: post #117

Earlier quoted context omitted.

Yes, but so is telling if a photo contains a dog or understanding sentiment in a paragraph of text. Complexity isn't quite the issue, I think it is that there is a distinction between the type of reasoning which these models have learnt and that which is necessary for concrete mathematical reasoning.

The models do not reason. They have learned associations, because these associations have appeared in their training sets.

They also generalise and categorise and perhaps even form abstractions based on those associations. Those are the beginnings of reasoning.

I expect that as the models grow more complicated so will their reasoning ability.

Re: Vision language models are blind

#194

Post by mr_toad »

jordan_bonecut wrote: post #116

This is an interesting article and goes along with how I understand how such models interpret input data. I'm not sure I would characterize the results as blurry vision, but maybe an inability to process what they see in a concrete manner. All the LLMs and multi-modal models I've seen lack concrete reasoning. For instance, ask ChatGPT to perform 2 tasks, to summarize a chunk of text and to count how many words are in…

Counting is hard, even for humans.

A child will start to speak at around the age of one, but most will be about two before they start to count. And it is even longer (maybe the age of three to four) before they understand cardinality and can reliably follow “simple” instructions like “bring me four blocks”.

And basic arithmetic without counting on their fingers is usually not picked up until they are around six or seven.

Re: Vision language models are blind

#196

Post by TeMPOraL »

foldr wrote: post #182
TeMPOraL wrote: post #165

Earlier quoted context omitted.

They're hardly being advertised or sold on that premise. They advertise and sell themselves, because people try them out and find out they work , and tell their friends and/or audiences. ChatGPT is probably the single biggest bona-fide organic marketing success story in recorded history.

This is fantastic news for software engineers. Turns out that all those execs who've decided to incorporate AI into their product strategy have already tried it out and ensured that it will actually work.

Who cares about execs? They know they work, but for them "works" is defined as "makes them money", not "does anything useful".

I'm talking about regular people, who actually use these tools for productive use, and can tell the models are up to tasks previously unachievable.

Re: Vision language models are blind

#197

Post by foldr »

TeMPOraL wrote: post #196
foldr wrote: post #182

Earlier quoted context omitted.

This is fantastic news for software engineers. Turns out that all those execs who've decided to incorporate AI into their product strategy have already tried it out and ensured that it will actually work.

Who cares about execs? They know they work, but for them "works" is defined as "makes them money", not "does anything useful". I'm talking about regular people, who actually use these tools for productive use, and can tell the models are up to tasks previously unachievable.

Execs are important in the context of a discussion of how LLMs are advertised and sold.

Re: Vision language models are blind

#198

Post by visarga »

dheera wrote: post #84
visarga wrote: post #53

Earlier quoted context omitted.

> An embedding isn't conceptually that much different from feeding a 1024-word description of an image instead of the actual image. An embedding needs less words. You can embed individual words, phrases, like a whole prompt and longer paragraphs. You don't need 1024 words for a text embed. For example a famous library is called Sentence BERT (sbert). When you embed images on the other hand, you cut them up into littl…

> needs less words Yes I'm aware of this, and work in ML -- the thing is embeddings are not designed for faithful image reconstruction, and aren't even trained that way. You can easily find two images that have substantially similar CLIP (or whatever) embeddings that are visually very different. If you query the LLM about that difference, the LLM wouldn't even have the information to differentiate answers for the two…

I remembered about a paper that sheds light on this issue. An embedding can store/recover exactly a short sentence:

> a multi step method that iteratively corrects and re embeds text is able to recover 92% of 32-token text inputs exactly

https://arxiv.org/abs/2310.06816

So it's probably 1 sentence == 1 embedding

Re: Vision language models are blind

#199

Post by MagicMoonlight »

jetrink wrote: post #65

I had a remarkable experience with GPT-4o yesterday. Our garage door started to fall down recently, so I inspected it and found that our landlord had installed the wire rope clips incorrectly, leading to the torsion cables losing tension. I didn't know what that piece of hardware was called, so I asked ChatGPT and it identified the part as I expected it to. As a test, I asked if there was anything notable about the p…

To trace it through space it would need short term memory and the ability to think. It does not have it. It must therefore be guessing.

Re: Vision language models are blind

#200

Post by TeMPOraL »

ben_w wrote: post #190
foldr wrote: post #182

Earlier quoted context omitted.

This is fantastic news for software engineers. Turns out that all those execs who've decided to incorporate AI into their product strategy have already tried it out and ensured that it will actually work.

> Turns out that all those execs who've decided to incorporate AI into their product strategy have already tried it out and ensured that it will actually work. The 2-4-6 game comes to mind. They may well have verified the AI will work, but it's hard to learn the skill of thinking about how to falsify a belief.

You mean this one here? - https://mathforlove.com/lesson/2-4-6-puzzle/

Looking at the example patterns given:

  MATCH
  2, 4, 6
  8, 10, 12
  12, 14, 16
  20, 40, 60

  NOT MATCH
  10, 8, 6
If the answer is "numbers in ascending order", then this is a perfect illustration of synthetic vs. realistic examples. The numbers indeed fit that rule, so in theory, everything is fine. In practice, you'd be an ass to give such examples on a test, because they strongly hint the rule is more complex. Real data from a real process is almost never misleading in this way[0]. In fact, if you sampled such sequences from a real process, you'd be better off assuming the rule is "2k, 2(k+1), 2(k+2)", and treating the last example as some weird outlier.

Might sound like pointless nitpicking, but I think it's something to keep in mind wrt. generative AI models, because the way they're trained makes them biased towards reality and away from synthetic examples.

--

[0] - It could be if you have very, very bad luck with sampling. Like winning a lottery, except the prize sucks.

Post reply on HN