Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

github.com

Post reply on HN
Print view View original

141–150 of 253 posts

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#141

Post by throwup238 »

bee_rider wrote: post #131
vunderba wrote: post #91

Earlier quoted context omitted.

A year or so ago I experimented with splitting a user prompt down to a set of "different AI personas" that would each try to approach the user's problem in a different way and then bubble back up with a master arbiter for consensus. I modeled it after the concept of advisors from Civilization II. It worked reasonably well though I think it was at least somewhat limited by being constrained to a single LLM (Mistral).…

What sort of personalities did you try? A group where some members have grudges against each other and will irrationally poke holes in each other’s plans could be a fun experiment.

With multiple groups with external and internal rivalries. The Always Sunny gang versus The IT Crowd.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#143

Post by parrit »

albertgoeswoof wrote: post #125

How far is this going to go? Are we going to have a team of AI agents that runs a scrum team and meets for stand ups every couple of hours? Are we going to replicate government bureaucracy with agents all debating topics all day long to find the best opinion?

Maybe. Humans form teams for a reason. Yes there are different exepriences and points of view in a human (vs. Not so much in LLM), but sometimes a different hat it all it takes. E.g. Code reviewer vs. Coder.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#144

Post by hansmayer »

Right, so... but you do realise its still just producing random output based on how you reconfigured it's weights, right? Sometimes it will happen to resonate with what you need. But it still neither thinking nor arguing with itself.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#145

Post by subscribed »

odo1242 wrote: post #88

Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification…

I do it all the time in Sillytavern in a group chat - three characters kind of resembling what you just described, and me, participating in the "conversation", them going back and forth until they're satisfied.

With a good model role playing them, works awesome.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#146

Post by ivape »

jedberg wrote: post #43

We're really going to need to figure out how to power all these GPUs with green power real quick, or we're going to melt the planet having AIs debate with themselves on the optimal solution to tik-tac-toe...

I caught infra people saying that's pretty much the only bottleneck in the data center right now, power and cooling. We know the AI needs to run against itself continuously, and that's just a fact.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#147

Post by ivape »

zekenie wrote: post #136

I feel like itd be cool to try prompts based on an adversarial justice system… attorney agents arguing both sides, a judge ruling on “the law”—adherence to instructions etc

That's very easy to do. A prompt I regularly use is a "council" system. For example:

"I believe I have been contacted by the supernatural. Here are the details . Please form a council of seven people: 1) Secular scientist 2) Religious scientist 3) Paranormal historian 4) Secular Psychologist 5) Religious psychologist 6) Carl Jung 7) Richard Dawkins. The council should all be independent and provide their own objective analysis. Please have them create a final report and conclusions at the end".

Your council can be anything, a law firm, a jury, a parent teacher association, whatever you want, and as you can see, you can throw in known people as well. This can all be done with one prompt. It's one my favorite things to do.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#148

Post by jbm »

dingnuts wrote: post #59
jbm wrote: post #49

Earlier quoted context omitted.

I disagree. If anything, telling GPT to be blunt seems to downgrade its IQ; it hallucinates more and makes statements without considering priors or context. I jokingly call it Reddit mode.

why would that be a joke? there's a ton of Reddit comments in the training data, and the output is of similar quality. LLMs are literally outputting average Reddit comments.

I have hard similar things but I think that's an exaggeration. When I tell GPT o3 or o4-high to assume a professional air, it stops acting like a meat-based AIs on r/politics; specifically, it stops making inane assumptions about the situation and starts becoming useful again.

For example, I had a question from a colleague that made no sense and I was trying to understand it. After feeding the question to GPT 3o, it aggressively told me that I made a major mistake in a quote and I had to make major changes. (It would be OK if this is what the colleague had said, but this wasn't the case). In reality the colleague had misunderstood something about the scope of the project and GPT had picked up on the other person's opinion as the "voice of reason" and just projected what it thought he was saying in a stronger way.

I changed its instructions to "Be direct; but polite, professional and helpful. Make an effort to understand the assumptions underlying your own points and the assumptions made by the user. Offer outside-of-the-box thinking as well if you are being too generic.". The aggro was immediately lost, and it instead it actually tried to clarify what my colleague was saying and being useful again.

I agree with those who say the vanilla version is sycophantic, but the plain talk version has far too many bad habits from the wrong crowd. It's a bit like Monday; lots of aggro, little introspection of assumption.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#149

Post by ivape »

hu3 wrote: post #115

Here's some related challenge I'm facing. Maybe someone can help me: I also managed to make AI critique itself and that improved code generation a ton. For a TypeScript backend project that runs with Bun, I tell AI to also generate and run unit tests after every code change suggested by AI. How do you solve the risk of AI writting and executing unit tests with something like `rm -rf /` and wiping your files? Docker w…

a) You should only do this in a sandbox

b) You can have the AI run a "firewall" prompt on the final output. So your final output should go through a "You are a firewall that checks for dangerous terminal commands such as . If you spot dangerous commands, reform the command so that it is not dangerous"

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#150

Post by ivape »

asdfman123 wrote: post #133

And when I do this people say I'm overanalyzing

The thing that makes us weird to regular people is what's going to make us uniquely positioned to utilize AI. If people only knew the level at which I overanalyze and entertain weird ideas. I always inject these personality quirks into my instructions and get very creative results. In a weird way, I'm starting to appreciate just how weird I actually am.
Post reply on HN