Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Emperor's New LLM

dayafter.substack.com

Post reply on HN
Print view View original

1–10 of 65 posts

Re: The Emperor's New LLM

#2

Post by Havoc »

I wonder whether that means routinely asking the reverse is the more useful feedback then.

If there is a bias towards agreeing then asking “is shit on a stick a terrible idea” and then it agrees but will tell you why

Re: The Emperor's New LLM

#3

Post by kelseyfrog »

Havoc wrote: post #2

I wonder whether that means routinely asking the reverse is the more useful feedback then. If there is a bias towards agreeing then asking “is shit on a stick a terrible idea” and then it agrees but will tell you why

In my experience starting prompts with "Evaluate and analyze.." are better than either at reducing bias, but unfortunately, once OpenAI touched the sycophantic mains, the fear of lingering flattery doesn't go away.

Re: The Emperor's New LLM

#5

Post by kypro »

I also find when it disagrees with you it does so in a really patronising way.

In my experience it will always first affirm me for having my own opinions, but then go on to explain why I'm wrong as if I'm a child or idiot – often by making appeals to authority or emotion to "disprove" me.

I wish they were designed to not have opinions on things. Just give me the data and explain why most people disagree with me without implying I'm some uneducated idiot because I don't 100% align with what most people think on a certain topic.

I always thought this would be one of the benefits of AI... That it would be more interested in assigning probabilities to truth statements given current data, rather than resolving on a single position in the way humans do. Instead LLMs seem to be much more opinionated and less rationally so than most humans.

Re: The Emperor's New LLM

#7

Post by sysmax »

>The same kind of bias keeps resurfacing in every major system: Claude, Gemini, Llama, clearly this isn’t just an OpenAI problem, it’s an LLM problem.

It's not an LLM problem, it's a problem of how people use it. It feels natural to have a sequential conversation, so people do that, and get frustrated. A much more powerful way is parallel: ask LLM to solve a problem. In a parallel window, repeat your question and the previous answer and ask to outline 10 potential problems. Pick which ones appear valid, ask to elaborate. Pick your shortlist, ask yet another LLM thread to "patch" the original reply with these criticisms, then continue the original conversation with a "patched" reply.

LLMs can can't tell legitimate concerns from nonsensical ones. But if you, the user, do, they will pick it up and do all the legwork.

Re: The Emperor's New LLM

#8

Post by rainonmoon »

kypro wrote: post #5

I also find when it disagrees with you it does so in a really patronising way. In my experience it will always first affirm me for having my own opinions, but then go on to explain why I'm wrong as if I'm a child or idiot – often by making appeals to authority or emotion to "disprove" me. I wish they were designed to not have opinions on things. Just give me the data and explain why most people disagree with me witho…

I'd be curious to know what your success rate is with altering the system prompt. I'd be surprised if this wasn't more of an issue with the application layer, and therefore easily modifiable, than the LLM.

Re: The Emperor's New LLM

#9

Post by thinkling »

sysmax wrote: post #7

>The same kind of bias keeps resurfacing in every major system: Claude, Gemini, Llama, clearly this isn’t just an OpenAI problem, it’s an LLM problem. It's not an LLM problem, it's a problem of how people use it. It feels natural to have a sequential conversation, so people do that, and get frustrated. A much more powerful way is parallel: ask LLM to solve a problem. In a parallel window, repeat your question and the…

You're saying roughly "you can't trust the first answer from an LLM but if you run it through enough times, the results will converge on something good". This, plus all the hoo-hah about prompt engineering, seem like clear signals that the "AI" in LLMs is not actually very intelligent (yet). It confirms the criticism.
Post reply on HN