Live data from Hacker News

Are LLMs able to notice the “gorilla in the data”?

chiraaggohel.com

191–200 of 207 posts

Re: Are LLMs able to notice the “gorilla in the data”?

#191

Earlier quoted context omitted.

That's a feature that would need to be implemented. There's no reason to think it could look at the image of the plot it generated automatically, but feeding it the image it generated back to it is no different to if it did view it automatically

The point of telling it to explore the data is so I don't have to think of every angle myself. Humans can get an understanding from visuals that LLMs can't match, apparently, even without gimmicks.

[deleted]

Re: Are LLMs able to notice the “gorilla in the data”?

#192

Earlier quoted context omitted.

That's a feature that would need to be implemented. There's no reason to think it could look at the image of the plot it generated automatically, but feeding it the image it generated back to it is no different to if it did view it automatically

The point of telling it to explore the data is so I don't have to think of every angle myself. Humans can get an understanding from visuals that LLMs can't match, apparently, even without gimmicks.

The llm is able to see the gorilla when shown the image in the same way you would show a human an image.

Imagine if you gave someone the raw data and told them to write code to graph the output but on to a screen they couldn't see. They would not be able to tell you it's a gorilla until you turn the monitor around and show them.

Humans are still better at seeing the image, sure (for now), but the llm is a tool with certain features and abilities. You can't make up a scenario that is misusing the tool and then pretend that it doesn't work - especially when it seems you want it to use it without applying your own brain power to the process

And to be clear, I'm open to criticism of llms and exploration of their limitations - but I'm tired of hearing complaints that amount to PEBKAC.

Re: Are LLMs able to notice the “gorilla in the data”?

#193

Earlier quoted context omitted.

The point of telling it to explore the data is so I don't have to think of every angle myself. Humans can get an understanding from visuals that LLMs can't match, apparently, even without gimmicks.

The llm is able to see the gorilla when shown the image in the same way you would show a human an image. Imagine if you gave someone the raw data and told them to write code to graph the output but on to a screen they couldn't see. They would not be able to tell you it's a gorilla until you turn the monitor around and show them. Humans are still better at seeing the image, sure (for now), but the llm is a tool with c…

When I tell a human to analyze the data, I sure don't expect them to interpret it as "write code to graph it to a screen you can't see". You found the problem but glossed right over it.

> misusing the tool and then pretend that it doesn't work

It was told to analyze and then it did a bad job of analyzing. I don't care if an LLM expert expects this already, it's worth pointing out to everyone else. It's not PEBKAC.

Re: Are LLMs able to notice the “gorilla in the data”?

#194

Earlier quoted context omitted.

The difference is that on that episode, the AI was actually capable of thinking. Asimov has an story like that too.

The Asimov story it reminded me of was The Profession, though that one is not really about AI - but it is about original ideas and the kinds of people that have them. I find the LLM dismissals somewhat tedious for most of the people making them half of humanity wouldn't meet their standards.

> half of humanity wouldn't meet their standards.

All anti ai sentiment as pertains to personhood that I've ever interacted with (and it was a lot, in academia) boils down to arguments for the soul. It is really tedious and before I spoke to people about it it probably wouldn't have passed my turing test. Sadly even very smart people may be very stupid and even in a place of learning a teacher will respect that (no matter how dumb or puerile), more than likely they think the exact same thing.

Re: Are LLMs able to notice the “gorilla in the data”?

#195

Earlier quoted context omitted.

The llm is able to see the gorilla when shown the image in the same way you would show a human an image. Imagine if you gave someone the raw data and told them to write code to graph the output but on to a screen they couldn't see. They would not be able to tell you it's a gorilla until you turn the monitor around and show them. Humans are still better at seeing the image, sure (for now), but the llm is a tool with c…

When I tell a human to analyze the data, I sure don't expect them to interpret it as "write code to graph it to a screen you can't see". You found the problem but glossed right over it. > misusing the tool and then pretend that it doesn't work It was told to analyze and then it did a bad job of analyzing. I don't care if an LLM expert expects this already, it's worth pointing out to everyone else. It's not PEBKAC.

If a user misunderstands the purpose and value of a tool, this is PEBKAC.

Re: Are LLMs able to notice the “gorilla in the data”?

#196

Earlier quoted context omitted.

Just imagining an episode of Star Trek where the inhabitants of a planet have been failing to progress in warp drive tech for several generations. The team beams down to discover that society's tech stopped progressing when they became addicted to pentesting their LLM for intelligence, only to then immediately patch the LLM in order to pass each particular pentest that it failed. Now the society's time and energy has…

There actually is an episode of TNG similar to that. The society stopped being able to think for themselves, because the AI did all their thinking for them. Anything the AI didn’t know how to do, they didn’t know how to do. It was in season 1 or season 2.

To add to the roll call of similar plotlines, there is also Zardoz. It's a hoot.

Re: Are LLMs able to notice the “gorilla in the data”?

#197
post #11

Earlier quoted context omitted.

Claude 3.5 Sonnet is much better at it: https://claude.site/artifacts/ad1b544f-4d1b-4fc2-9862-d6438e... But I guess GPT-4o results are more funny to look at.

Two pink squiggles is better than the hundred listed in the OP link?

Yeah because the ones from GPT-4o are not unicorns (in the vast majority of cases), while the Claude ones are unicorns.

Re: Are LLMs able to notice the “gorilla in the data”?

#198

Earlier quoted context omitted.

When I tell a human to analyze the data, I sure don't expect them to interpret it as "write code to graph it to a screen you can't see". You found the problem but glossed right over it. > misusing the tool and then pretend that it doesn't work It was told to analyze and then it did a bad job of analyzing. I don't care if an LLM expert expects this already, it's worth pointing out to everyone else. It's not PEBKAC.

If a user misunderstands the purpose and value of a tool, this is PEBKAC.

The tool doesn't succeed at a reasonable task that humans can do. That's not PEBKAC, and warning people about it is a good thing.

This type of analysis is not outside the purpose of the tool. You're making excuses at this point. Do you really think it would be wrong to add that capability in the future?

It's a technical limitation, one that is far from obvious.

Re: Are LLMs able to notice the “gorilla in the data”?

#199

Earlier quoted context omitted.

If a user misunderstands the purpose and value of a tool, this is PEBKAC.

The tool doesn't succeed at a reasonable task that humans can do. That's not PEBKAC, and warning people about it is a good thing. This type of analysis is not outside the purpose of the tool. You're making excuses at this point. Do you really think it would be wrong to add that capability in the future? It's a technical limitation, one that is far from obvious.

I think you think the llm is a magic box with an intelligent being inside that can magically do whatever you want it to, somehow. It is software. It has capabilities and limitations. Learn them, and use it appropriately. Or don't, you don't have to use it. But don't expect it to just do whatever you think it should do.

Re: Are LLMs able to notice the “gorilla in the data”?

#200

Earlier quoted context omitted.

The people that coined that term actually liked season 3 but I think they still don't recommend it because the hack fraud that directed the first two seasons ruined Star Trek forever. Just like JJ. And nutrek

can you say something redeemable about the plot that doesnt invoke "brought back the cast"?

no, because i didn't watch it. I've never really been into star trek. i watched a few of the movies - nemesis and the one before it, the jj one(s) and then i was done.

If i can remember the review it was "this is a capstone on TNG and probably the entire franchise for most of the older fans" and the first two seasons being disregarded, seasons 3 is "passable"

Post reply on HN