Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vision language models are blind

vlmsareblind.github.io

Post reply on HN
Print view View original

161–170 of 202 posts

Re: Vision language models are blind

#161

Post by [deleted] »

benreesman wrote: post #94
SomaticPirate wrote: post #18

Earlier quoted context omitted.

There are quite a few "ai apologists" in the comments but I think the title is fair when these models are marketed towards low vision people ("Be my eyes" https://www.youtube.com/watch?v=Zq710AKC1gg ) as the equivalent to human vision. These models are implied to be human level equivalents when they are not. This paper demonstrates that there are still some major gaps where simple problems confound the models in unex…

If we're throwing "citation needed" tags on stuff, how about the first sentence? "Large language models with vision capabilities (VLMs), e.g., GPT-4o and Gemini-1.5 Pro are powering countless image-text processing applications" I don't know how many a "countless" is, but I think we've gotten really sloppy in terms of what counts for LLMs as a demonstrated, durable win in a concrete task attached to well-measured outc…

[deleted]

Re: Vision language models are blind

#162

Post by TeMPOraL »

ClumsyPilot wrote: post #148
godelski wrote: post #87

Earlier quoted context omitted.

> Did they try to probe that hypothesis at all? I think this is a communication issue and you're being a bit myopic in your interpretation. It is clearly an analogy meant for communication and is not an actual hypothesis. Sure, they could have used a better analogy and they could have done other tests, but the paper still counters quite common claims (from researchers) about VLMs. > I could (well actually I can't) sh…

> Second, so what if a test was designed to trick up a model? Shouldn't we be determining when and where models fail? Is that not a critical question in understanding how to use them properly? People are rushing to build this AI into all kinds of products, and they actively don’t want to know where the problems are. The real world outside is designed to trip up the model. Strange things happen all the time. Because s…

> The real world outside is designed to trip up the model. Strange things happen all the time.

Counterpoint: real world is heavily sanitized towards things that don't trip human visual perception up too much, or otherwise inconvenience us. ML models are trained on that, and for that. They're not trained for dealing with synthetic images, that couldn't possibly exist in reality, and designed to trip visual processing algorithms up.

Also:

> People are rushing to build this AI into all kinds of products, and they actively don’t want to know where the problems are.

Glass half-full (of gasoline) take: those products will trip over real-world problems, identifying them in the process, and the models will get better walking over the corpses of failed AI-get-rich-quick companies. The people involved may not want to know where the problems are, but by deploying the models, they'll reveal those problems to all.

> Because software developers have no governing body, no oaths of ethics and no spine someone will end up dead in a ditch from malfunctioning AI.

That, unfortunately, I 100% agree with. Though AI isn't special here - not giving a fuck kills people regardless of the complexity of software involved.

Re: Vision language models are blind

#163

Post by Foobar8568 »

youssefabdelm wrote: post #107
Lerc wrote: post #106

Earlier quoted context omitted.

I disagree. I think the title, abstract, and conclusion not only misrepresents the state of the models but it misrepresents Thier own findings. They have identified a class of problems that the models perform poorly at and have given a good description of the failure. They portray this as a representative example of the behaviour in general. This has not been shown and is probably not true. I don't think that models…

Exactly... I've found GPT-4o to be good at OCR for instance... doesn't seem "blind" to me.

I know that GPT-4o is fairly poor to recognize music sheets and notes. Totally off the marks, more often than not, even the first note is not recognize on a first week solfège book.

So unless I missed something but as far as I am concerned, they are optimized for benchmarks.

So while I enjoy gen AI, image-to-text is highly subpart.

Re: Vision language models are blind

#164

Post by youssefabdelm »

Foobar8568 wrote: post #163
youssefabdelm wrote: post #107

Earlier quoted context omitted.

Exactly... I've found GPT-4o to be good at OCR for instance... doesn't seem "blind" to me.

I know that GPT-4o is fairly poor to recognize music sheets and notes. Totally off the marks, more often than not, even the first note is not recognize on a first week solfège book. So unless I missed something but as far as I am concerned, they are optimized for benchmarks. So while I enjoy gen AI, image-to-text is highly subpart.

Useful to know, thank you!

Re: Vision language models are blind

#165

Post by TeMPOraL »

cs702 wrote: post #13
tensor wrote: post #9

Earlier quoted context omitted.

I don't see how this is "embarrassing" in the slightest. These models are not human brains, and the fact that people equate them with human brains is an embarrassing failure of the humans more than anything about the models. It's entirely unsurprising that there are numerous cases that these models can't handle that are "obvious to humans." Machine learning has had this property since its invention and it's a classic…

It is embarrassingly, shockingly bad, because these models are advertised and sold as being capable of understanding images. Evidently, all these models still fall short.

They're hardly being advertised or sold on that premise. They advertise and sell themselves, because people try them out and find out they work, and tell their friends and/or audiences. ChatGPT is probably the single biggest bona-fide organic marketing success story in recorded history.

Re: Vision language models are blind

#166

Post by _flux »

itkovian_ wrote: post #139
sweezyjeezy wrote: post #8

Entertaining, but I think the conclusion is way off. > their vision is, at best, like that of a person with myopia seeing fine details as blurry is a crazy thing to write in an abstract. Did they try to probe that hypothesis at all? I could (well actually I can't) share some examples from my job of GPT-4v doing some pretty difficult fine-grained visual tasks that invalidate this. Personally, I rate this paper [1], wh…

I think gpt4o is probably doing some ocr as preprocessing. It's not really controversial to say the vmls today don't pick up fine grained details - we all know this. Can just look at the output of a vae to know this is true.

Why do you think it's probable? The much smaller llava that I can run in my consumer GPU can also do "OCR", yet I don't believe anyone has hidden any OCR engine inside llama.cpp.

Re: Vision language models are blind

#167

Post by prmoustache »

youssefabdelm wrote: post #107
Lerc wrote: post #106

Earlier quoted context omitted.

I disagree. I think the title, abstract, and conclusion not only misrepresents the state of the models but it misrepresents Thier own findings. They have identified a class of problems that the models perform poorly at and have given a good description of the failure. They portray this as a representative example of the behaviour in general. This has not been shown and is probably not true. I don't think that models…

Exactly... I've found GPT-4o to be good at OCR for instance... doesn't seem "blind" to me.

Well maybe not blind but the analogy with myopia might stand.

For exemple in the case of OCR, a person with myopia will usually be able to make up letters and words even without his glasses based on his expectation (similar to vlm training) of seeing letters and words in, say, a sign. He might not see them all clearly and do some errors but might recognize some letters easily and make up the rest based on context, words recognition, etc. Basically experience.

I also have a funny anecdote about my partner, which has sever myopia, who once found herself outside her house without her glasses on, and saw something on the grass right in front. She told her then brother in law "look, a squirrel" Only for the "squirrel" to take off while shouting its typical caws. It was a crow. This is typical of VLM's hallucinations.

Re: Vision language models are blind

#168

Post by empiricus »

jordan_bonecut wrote: post #116

This is an interesting article and goes along with how I understand how such models interpret input data. I'm not sure I would characterize the results as blurry vision, but maybe an inability to process what they see in a concrete manner. All the LLMs and multi-modal models I've seen lack concrete reasoning. For instance, ask ChatGPT to perform 2 tasks, to summarize a chunk of text and to count how many words are in…

I hope you are aware of the fact that LLMs does not have direct access to the stream of words/characters. It is one of the most basic things to know about their implementation.

Re: Vision language models are blind

#169

Post by pegasus »

TeMPOraL wrote: post #159
pixl97 wrote: post #54

Earlier quoted context omitted.

Yea, really if you look at human learning/seeing/acting there is a feedback loop that LLM for example isn't able to complete and train on. You see an object. First you have to learn how to control all your body functions to move toward it and grasp it. This teaches you about the 3 dimensional world and things like gravity. You may not know the terms, but it is baked in your learning model. After you get an object you…

Embodied is useful, but I think not necessary even if you need learning in a 3D environment. Synthesized embodiment should be enough. While in some cases[0] it may have problems with fidelity, simulating embodied experience in silico scales much better, and more importantly, we have control over time flow . Humans always learn in real-time, while with simulated embodiment, we could cram years of subjective-time exper…

> Like if you plugged a 3D game engine into the training loop

Isn't this what synthesized embodiment basically always is? As long as the application of the resulting technology is in a restricted, well controlled environment, as is the case for example for an assembly-line robot, this is a great strategy. But I expect fidelity problems will make this technique ultimately a bad idea for anything that's supposed to interact with humans. Like self-driving cars, for example. Unless, again, those self-driving cars are segregated in dedicated lanes.

Re: Vision language models are blind

#170

Post by godelski »

TeMPOraL wrote: post #162
ClumsyPilot wrote: post #148

Earlier quoted context omitted.

> Second, so what if a test was designed to trick up a model? Shouldn't we be determining when and where models fail? Is that not a critical question in understanding how to use them properly? People are rushing to build this AI into all kinds of products, and they actively don’t want to know where the problems are. The real world outside is designed to trip up the model. Strange things happen all the time. Because s…

> The real world outside is designed to trip up the model. Strange things happen all the time. Counterpoint: real world is heavily sanitized towards things that don't trip human visual perception up too much, or otherwise inconvenience us. ML models are trained on that, and for that. They're not trained for dealing with synthetic images, that couldn't possibly exist in reality, and designed to trip visual processing…

> They're not trained for dealing with synthetic images, that couldn't possibly exist in reality, and designed to trip visual processing algorithms up

Neither of these claims are true. ML is highly trained on synthetic images. In fact, synthetic data generation is the way forward for the scale is all you need people. And there are also loads of synthetic images out in the wild. Everything from line art to abstract nonsense. Just take a walk down town near the bars.

> not giving a fuck kills people regardless of the complexity of software involved.

What has me the most frustrated is that this "move fast break things and don't bother cleaning up" attitude is not only common in industry but also in academia. But these two are incredibly intertwined these days and it's hard to publish without support from industry because people only evaluate on benchmarks. And if you're going to hack your benchmarks, you just throw a shit ton of compute at it. Who cares where the metrics fail?

Post reply on HN