Live data from Hacker News

Are LLMs able to notice the “gorilla in the data”?

chiraaggohel.com

41–50 of 207 posts

Re: Are LLMs able to notice the “gorilla in the data”?

#42
post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

Even beyond training I asked an llm to generate me an image of a corndog. It would only give me hot dogs until I described how a corndog is made.

Not the end of the world but it does seem like AI gets fixated, like people, and can’t see anything else.

Re: Are LLMs able to notice the “gorilla in the data”?

#43
post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

I've had a Nest camera in my living room for years just to keep an eye on our dogs while we're away from home. One of the dogs, a basset hound/border collie mix, often howls and makes "squeeing" noises while we're away. Nest (or Google now, I suppose) without fail thinks that this is actually a person talking in my living room and sends us notifications alerting us to this fact. If he moves around, Nest thinks it's a person moving in my living room.

It has no problem identifying our other two dogs as actual dogs who bark and move like dogs.

Re: Are LLMs able to notice the “gorilla in the data”?

#44
post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

Obviously you need to place a fiducial mark on your cat.

Re: Are LLMs able to notice the “gorilla in the data”?

#45
post #33

Earlier quoted context omitted.

Please read TFA. The conclusion of the article isn't nearly so simplistic, they're just suggesting that you have to be aware of the natural strengths and weaknesses of LLMs, even multi modal ones particularly around visual pattern recognition vs quantitative pattern recognition. And yes, the idea that the initial context can sometimes predispose the LLM to consider things in a more narrow manner than a user might oth…

The title of the article is "Your AI Can't See Gorillas". That seems demonstrably false. The article says: > Furthermore, their data analysis capabilities seem to focus much more on quantitative metrics and summary statistics, and less on the visual structure of the data Again, this seems false - or, at best, misleading. I had no problem getting AI to focus on visual structure of the data without any tricks. A more f…

you knew that there was a visual gag in there before asking it to.

if you didnt know it was there, and took a look at only the text output, the llm would not have found it to tell you its there

Re: Are LLMs able to notice the “gorilla in the data”?

#46
post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

I'd like to see a photo of said cat, for science of course :)

Re: Are LLMs able to notice the “gorilla in the data”?

#47
post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

[deleted]

Re: Are LLMs able to notice the “gorilla in the data”?

#48
post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

I have a Tapo camera which from all the cats parading from my yard, only one black cat is recognized as a person. Not other cats. Not even other black cats. It makes you think.

Re: Are LLMs able to notice the “gorilla in the data”?

#49
I had a recent similar experience with chat gpt and a gorilla. I was designing a rather complicated algorithm so I wrote out all the steps in words. I then asked chatgpt to verify that it made sense. It said it was well thought out, logical etc. My colleague didn't believe that it was really reading it properly so I inserted a step in the middle "and then a gorilla appears" and asked it again. Sure enough, it again came back saying it was well thought out etc. When I questioned it on the gorilla, it merely replied saying that it thought it was meant to be there, that it was a technical term or a codename for something...

Re: Are LLMs able to notice the “gorilla in the data”?

#50
Seems like the specific goal post of gorilla was chosen in order to obtain the outcome to write the paper they wanted and rather uninteresting compared to determining at what point does the AI start to see shapes in the data. Could the AI see a line, curve, square, or an umbrella? If AI can't see a square why would we expect it to see a gorilla?
Post reply on HN