Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are LLMs able to notice the “gorilla in the data”?

chiraaggohel.com

Post reply on HN
Print view View original

141–150 of 207 posts

Re: Are LLMs able to notice the “gorilla in the data”?

#141

Post by blharr »

wyldfire wrote: post #118
PunchTornado wrote: post #113

Earlier quoted context omitted.

In the article they also give the png

> When a png is directly uploaded, the model is better able to notice that some strange pattern is present in the data. However, it still does not recognize the pattern as a gorilla. I wonder if the conversation context unfairly weighed the new impression towards the previous interpretation.

I'm curious if there's any good resources on this, but I've noticed including the conversation context makes the responses drastically worse quality in my experience. It's gotten to a point whe

Re: Are LLMs able to notice the “gorilla in the data”?

#142

Post by layer8 »

HappMacDonald wrote: post #136
phyzome wrote: post #108

Earlier quoted context omitted.

« saying that it thought » This is, quite literally, not something that ChatGPT has the capability to do -- reporting on its thought process, that is. This is a hallucination.

Most human self-reflection and to an extent even memory is similarly post-facto, however.

Humans do tend to remember thoughts they had while speaking, thoughts that go beyond what they said. LLMs don’t have any memory of their internal states beyond what they output.

(Of course, chain-of-thought architectures can hide part of the output from the user, and you could declare that as internal processes that the LLM does “remember” in the further course if the chat.)

Re: Are LLMs able to notice the “gorilla in the data”?

#143

Post by codemac »

mariofilho wrote: post #66

I uploaded the image to Gemini 2.0 Flash Thinking 01 21 and asked: “ Here is a steps vs bmi plot. What do you notice?” Part of the answer: “Monkey Shape: The most striking feature of this plot is that the data points are arranged to form the shape of a monkey. This is not a typical scatter plot where you'd expect to see trends or correlations between variables in a statistical sense. Instead, it appears to be a creat…

o1 said: It appears the points have been deliberately arranged to form a stylized figure rather than showing a conventional trend. In other words, there is no obvious linear or monotonic relationship between steps and BMI, and the red‐blue (female–male) split is intermixed throughout the shape. From this visualization alone, one cannot meaningfully conclude a correlation or difference in BMI by gender or by step count. Instead, the data are displayed in a pattern that resembles a cartoon character, suggesting the layout is artistic or contrived rather than reflecting a standard distribution.

o1 pro: It appears the data have been deliberately arranged to form a stylized humanoid figure. In terms of actual trends, there is no obvious linear or nonlinear correlation between steps and BMI, and the female–male color split seems fairly uniform across the shape. Visually, BMI spans roughly 15–30, while steps extend up to about 15,000, with no clear clustering or separation by gender. Within reasonable confidence (e.g., an estimated R² near zero if one were to attempt a linear fit), there is no discernible predictive relationship between steps and BMI in this plot.

Re: Are LLMs able to notice the “gorilla in the data”?

#144

Post by kwerk »

sillysaurusx wrote: post #85
jancsika wrote: post #84

Earlier quoted context omitted.

Just imagining an episode of Star Trek where the inhabitants of a planet have been failing to progress in warp drive tech for several generations. The team beams down to discover that society's tech stopped progressing when they became addicted to pentesting their LLM for intelligence, only to then immediately patch the LLM in order to pass each particular pentest that it failed. Now the society's time and energy has…

There actually is an episode of TNG similar to that. The society stopped being able to think for themselves, because the AI did all their thinking for them. Anything the AI didn’t know how to do, they didn’t know how to do. It was in season 1 or season 2.

I pasted your comment into mistral small latest, google, gpt 4o and gpt 4o with search. They all have a different answe, only the last gave a real episode but it said 11001001 in season 1. It said episode 15, it’s actually 14. But even that seems wrong.

Are they censored from showing this cautionary tale?? Hah.

Re: Are LLMs able to notice the “gorilla in the data”?

#145

Post by runjake »

coder543 wrote: post #56
runjake wrote: post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

How exactly are you trying to train and deploy this YOLO model? What kind of accuracy are you seeing against the validation set at the end of the training process?

Sorry, to answer your other question, this is via CodeProject.AI.

Re: Are LLMs able to notice the “gorilla in the data”?

#146

Post by runjake »

GaggiX wrote: post #30
runjake wrote: post #28

Only tangentially related to this story, I've been trying for months to train the YOLO models to recognize my Prussian blue cat, with its assorted white spots, as a cat rather than a dog or a person. However, it refuses to cooperate. It's maddening. As a result, I receive "There is a person at your front door" notifications at all hours of the night.

Something is very wrong if the model cannot tell the difference between a Prussian blue cat and a person. I imagine you have inserted in training data the images of the cat from the camera and in similar quantities of a person from the same camera.

I’ve wiped the local training for, and swapped YOLO model versions of number of times. It doesn’t make any difference.

In truth it’s only mildly annoying and makes me appreciate my cat’s quirkiness more.

Re: Are LLMs able to notice the “gorilla in the data”?

#147

Post by sinuhe69 »

This could serve as a subplot for a sci-fi story: alien are trying for decades to contact by sending out encoded messages. Because the first sentinels are powerful AI, and only analyzed the data pattern, humanity was ignorant of the messages for decades. Until one day, a young astronomer played with the data and asked the AI to plot and visualize in many different ways, just for fun and suddenly realized the hidden images encoded in the messages :)

Re: Are LLMs able to notice the “gorilla in the data”?

#148

Post by taberiand »

forgotusername6 wrote: post #49

I had a recent similar experience with chat gpt and a gorilla. I was designing a rather complicated algorithm so I wrote out all the steps in words. I then asked chatgpt to verify that it made sense. It said it was well thought out, logical etc. My colleague didn't believe that it was really reading it properly so I inserted a step in the middle "and then a gorilla appears" and asked it again. Sure enough, it again c…

I think you asked a yes bot to say yes to you. Did you set the context for the llm to ask it to be thorough and identify any unusual steps, ensure its feedback was critical and comprehensive etc etc? These tools don't work if you hold them wrong

Eg, from uploading the gorilla scatterplot to gpt4o and asking "What do you see?"

"The image is a scatter plot of "Steps vs BMI by Gender," where data points are color-coded:

Blue (x) for males

Red (x) for females

The data points are arranged in a way that forms an ASCII-art-style image of a "smirking monkey" with one hand raised. This suggests that the data may have been intentionally structured or manipulated to create this pattern.

Would you like me to analyze the raw data from the uploaded file? "

I have custom instructions that would influence its approach. And it does look more like a monkey than a gorilla to me

Re: Are LLMs able to notice the “gorilla in the data”?

#149

Post by taberiand »

Dylan16807 wrote: post #122
appleorchard46 wrote: post #91

These posts about X task LLMs fails at when you give it Y prompt are getting more and more silly. If you ask an AI to analyze some data, should the default behavior be to use that data to make various types of graphs, export said graphs, feed them back in to itself, then analyze the shapes of those graphs to see if they resemble an animal? Personally I would be very annoyed if I actually wanted a statistical analysis…

You don't think "Examine the data" and "Which other conclusions can you draw from the data?" are open-ended? And even when explicitly prompted to look at the plot, they only brush up against the data anomalies rather than properly analyzing the plot.

I tried it on gpt4o with an upload of the image and "What do you see?" as prompt and it said "monkey". So ymmv, these tools can't be evaluated with just a bunch of gotcha prompts and ignorance of how to use them effectively

Re: Are LLMs able to notice the “gorilla in the data”?

#150

Post by Dylan16807 »

taberiand wrote: post #149
Dylan16807 wrote: post #122

Earlier quoted context omitted.

You don't think "Examine the data" and "Which other conclusions can you draw from the data?" are open-ended? And even when explicitly prompted to look at the plot, they only brush up against the data anomalies rather than properly analyzing the plot.

I tried it on gpt4o with an upload of the image and "What do you see?" as prompt and it said "monkey". So ymmv, these tools can't be evaluated with just a bunch of gotcha prompts and ignorance of how to use them effectively

It's not a gotcha to give it the data points and ask it to analyze. Uploading this data in image form is effectively a leading question tuned to the specific data, and an analysis tool that needs that kind of leading question is not good at its job.
Post reply on HN