Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How much of AI's recent success is due to the Forer Effect?

shkspr.mobi

Post reply on HN
Print view View original

11–20 of 83 posts

Re: How much of AI's recent success is due to the Forer Effect?

#11

Post by Spivak »

> Some of them become emotional at having their personality revealed to them. Only then to be told that everyone gets the same results.

I think this is one of those cases where I don't think it's as much of a gotcha as the author is supposing because those tests act as a kind of mirror allowing you to project yourself onto the results and then see yourself in the reflection. It's why they're useful when you take them yourself as a self-reflection tool but not at all actionable when someone else tries to estimate you with them.

wink wink nudge nudge tarot works on the same principal

Re: How much of AI's recent success is due to the Forer Effect?

#12

Post by Willuminaughty »

Wow, this article really got me thinking! It's fascinating how AI, like ChatGPT, can create statements that feel so personal and unique to us, when in reality they're quite generic. It's a testament to the power of the Forer Effect, and how our brains are wired to find meaning and personal relevance in broad statements. It's a bit like reading a horoscope, isn't it? You can't help but feel like it's speaking directly to you. This definitely adds a new layer to how I perceive AI-generated content. It's not just about the technology's capability to generate human-like text, but also about our human tendency to find personal meaning in it.

Re: How much of AI's recent success is due to the Forer Effect?

#13

Post by choeger »

fourseventy wrote: post #5

The Forer Effect can't make chatGPT generate syntactically correct and functional code snippets that I use every day at work.

Let me make this very clear: Generating syntactically correct code in various languages that also looks plausible is no small feat. It is, in fact, extremely impressive and will certainly have an impact on SE.

But. Every single test I ran lead to functionally wrong designs from smallish memory errors in C (that hilariously ChatGPT was able to correct ND explain when pointed to) over misplaced/hallucinated methods in python APIs to completely hallucinated perl packages. It never gave me a useful answer.

I don't know if that tells us something about my work vs. your work or my approach to the model vs yours. But it definitely tells me that such a model cannot replace a developer.

Re: How much of AI's recent success is due to the Forer Effect?

#14

Post by juujian »

Interesting observation. I find it incredibly difficult to write queries for ChatGPT that are not biased. If you write a question mentioning two concept, the answer will almost without fail suggest a connection between the two. Whereas if you were to mention just one of them, the chance that the other concept appears in the response might be very small.

Re: How much of AI's recent success is due to the Forer Effect?

#15

Post by bjourne »

Willuminaughty wrote: post #12

Wow, this article really got me thinking! It's fascinating how AI, like ChatGPT, can create statements that feel so personal and unique to us, when in reality they're quite generic. It's a testament to the power of the Forer Effect, and how our brains are wired to find meaning and personal relevance in broad statements. It's a bit like reading a horoscope, isn't it? You can't help but feel like it's speaking directly…

You used ChatGPT to generate that comment, didn't you?

Re: How much of AI's recent success is due to the Forer Effect?

#16

Post by reesul »

I agree with the other comments that this is a fairly useless article. Perhaps I’m looking too much into this specific example, but I fail to see how the entirety of AI’s success (per this title) is misrepresented due to its inability to provide a horoscope (good example by another comment) that’s specific to some person in question.

We see a lot of articles that swing too far towards “AI will change everything!” just as much as “no, AI is not actually effective/meaningful!”. This is the latter. I’m surprised that anyone would genuinely try to use chatGPT/language LLMs this way.

I’m mainly enjoying chatGPT as a way to distill widespread information on the internet into a pseudo conversation. It’s great for learning new topics - my favorite is generating and explaining code snippets for languages/libraries I’m not familiar with.

Re: How much of AI's recent success is due to the Forer Effect?

#17

Post by JieJie »

jjk166 wrote: post #8

What fraction of people using AI are looking up opinions on themselves? While I'm sure someone has written a completely automated horoscope generator, in general I think these LLMs are being applied in situations where there would be no Forer Effect.

It's a time-honored tradition with new tech, asking it what it thinks about us.

"According to a study by the Pew Internet & American life project,[3] 47% of American adult Internet users have undertaken a vanity search in Google or another search engine. Some egosurf purely for entertainment, such as finding celebrities with the same name. However, many people egosurf as a means of online reputation management. Egosurfing can be used to find data spills, released information that is undesirable to have in the public eye. By searching one's own name in an online search engine, one can take on the perspective of a stranger attempting to find out personal information. Some egosurf in order to conceal personal images or information from potential employers, clients, identity thieves and the like. Similarly, some use egosurfing to maintain a positive public image and to achieve self-promotion."[0]

[0] https://en.wikipedia.org/wiki/Egosurfing

Re: How much of AI's recent success is due to the Forer Effect?

#18

Post by Filligree »

PaulHoule wrote: post #10
fourseventy wrote: post #5

The Forer Effect can't make chatGPT generate syntactically correct and functional code snippets that I use every day at work.

The odd thing is that some people report much better results than others. Most literature on the subject points out the importance of a "human in the loop" https://arxiv.org/abs/2304.13187 Generally, chatbots function in direct communication with humans and do not function autonomously. Thus they are dependent on not just human meaning-making but human thirst for meaning which will find it even when it isn't there.

> Generally, chatbots function in direct communication with humans and do not function autonomously. Thus they are dependent on not just human meaning-making but human thirst for meaning which will find it even when it isn't there.

The latter doesn't follow. They may just be dependent on good prompt design.

People who aren't used to the AIs, and don't know how to use them, get worse results. This shouldn't be a surprise; it's how every tool works.

Re: How much of AI's recent success is due to the Forer Effect?

#19

Post by scott_w »

Filligree wrote: post #7
fourseventy wrote: post #5

The Forer Effect can't make chatGPT generate syntactically correct and functional code snippets that I use every day at work.

The goalposts are moving so fast they're red-shifting.

That’s not really an issue when we’re talking about AI. I said in a separate comment on the topic: intelligence is such a complicated thing that we seem to only be able to define it by pointing to things and saying “that’s not it.”

If we didn’t move the goalposts, we’d have declared Stockfish to be full AI, despite it only being a chess-playing program, long ago.

Post reply on HN