Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

github.com

Post reply on HN
Print view View original

71–80 of 253 posts

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#71

Post by kridsdale1 »

irthomasthomas wrote: post #58
hnuser123456 wrote: post #2

I'm having a lot of fun experimenting with stuff like this. I'm trying to put together an unrealengine blueprints style graph editor to allow people to design workflows like this where you start with the user prompt input, which goes to one agent, which makes an initial attempt, and then that conversation history gets passed to another "agent" with a different system prompt telling it to be a harsh critic, but to als…

I think you can do most of this already with llm-consortium (maybe needs the llm-openrouter plugin with my pr merging) A consortium sends the same prompt to multiple models in parallel and the responses are all sent to one arbiter model which judges the model responses. The arbiter decides if more iterations are required. It can also be forced to iterate more until confidence-threshold or min-iterations. Now, using t…

Any links or names of example implementations of this?

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#72

Post by Lerc »

nonethewiser wrote: post #64
Lerc wrote: post #54

I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…

In theory couldnt this just be baked into a single adversarial model?

To an extent, but different models are better at different things.

That is something I'm also curious about. Given models (that use the same tokenisation) that are better at different things, would their be interesting things to find by analysing the logprobs for tokens generated from identical inputs (including cross feeding the generated token from one to another)

Surely there must be something notable at particular points when a model goes off on the wrong path.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#74

Post by mecsred »

eddieroger wrote: post #42
mecsred wrote: post #6

Earlier quoted context omitted.

_#_ has to be one of the worst word shortening schemes I've ever seen get widespread. It only works with a very small number of long-lived technologies, in which case they basically just get a nickname, "k8s" "i18n". It does not at all work for larger contexts. You're basically making someone solve a crossword (2 across, 10 letters with two filled in) just to parse your sentence.

It's just another form of any other jargon - unknown until you know it, and usually specific to the use case. I see k8s and i18n or a11y and I know exactly what they mean because at some point I learned it and it's part of the world I live in. Searching for stuff is how we learn, not solving crosswords.

Right, my complaint is that it only works like jargon, where you are just giving something a context-specific nickname. As a word shortening scheme, it's terrible. A world where many projects have names like s11g is a nightmare.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#75

Post by [deleted] »

Lerc wrote: post #54

I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…

[deleted]

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#77

Post by irthomasthomas »

kridsdale1 wrote: post #71
irthomasthomas wrote: post #58

Earlier quoted context omitted.

I think you can do most of this already with llm-consortium (maybe needs the llm-openrouter plugin with my pr merging) A consortium sends the same prompt to multiple models in parallel and the responses are all sent to one arbiter model which judges the model responses. The arbiter decides if more iterations are required. It can also be forced to iterate more until confidence-threshold or min-iterations. Now, using t…

Any links or names of example implementations of this?

https://github.com/irthomasthomas/llm-consortium

also, you aren't limited to cli. When you save a consortium it creates a model. You can then interact with a consortium as if it where a normal model (albeit slower and higher quality). You can then serve your custom models on an openai endpoint and use them with any chat client that supports custom openai endpoints.

The default behaviour is to output just the final synthesis, and this should conform to your user prompt. I recently added the ability to continue conversations with a consortium. In this case it only includes your user prompt and final synthesis in the conversation, so it mimics a normal chat, unlike running multiple iterations in the consortium, where full iteration history and arbiter responses are included.

UV tool install llm

llm install llm-consortium

llm install llm-model-gateway

llm consortium save qwen-gem-sonnet -m qwen3-32b -n 2 -m sonnet-3.7 -m gemini-2.5-pro --arbiter gemini-2.5-flash --confidence-threshold 95 --max-iterations 3

llm serve qwen-gem-sonnet

In this example I used -n 2 on the qwen model since it's so cheap we can include multiple instances of it in a consortium

Gemini flash works well as the arbiter for most prompts. However if your prompt has complex formatting requirements, then embedding that within an already complex consortium prompt often confuses it. In that case use gemini-2.5-pro for the arbiter. .

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#78

Post by z2 »

mparnisari wrote: post #51

So like rubber ducking for AI?

I would really like to see a fusion guidebook of mental tricks that work for humans and just as well for AI. Or humorously, perhaps prompt-engineering tricks that are also great mental hacks for better or clearer human thinking.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#79

Post by caseyy »

I tried something similar when Llama2 came out, pitting two assistants, who each believed the other is the user, against each other. Ultimately, it was the same model talking with itself. The system prompts for both had various instructions to disagree and criticise the opinion of the user. I provided the first message to get things started. Usually, it’s be along the lines of “nuclear proliferation is harmful to humanity”.

After 15 or so iterations, both assistants would keep repeating the same things and find agreement anyway. Sometimes, the chat became unhinged and useless, but 95/100 times, it was agreement.

Happy someone else made it work.

Post reply on HN