Did something similar (OverkiLLM) to this waayyyy back in August with open LLMs. I'm sure it'd work much better now:
Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
61–70 of 253 posts
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#62This seems to be different than I expected from the title. I thought it would be explicitly adversarial. 1. You are the assistant. Please answer the question directly. 2. You are the cross-examiner. The assistant is wrong. Explain why. 3. You are the assistant. The cross-examiner is wrong. Defend your claim. 4. You are a judge. Did either party make their case, or is another round of argumentation required? I haven't…
Chatgpt shares context between chats. I wonder how that impacts it?
It seems like a good approach though. What you dont want to do is ever suggest that its wrong yourself. Usually it will just assume it is wrong.
Actually what I find impressive is when I do this and it actually pushes back to defend itself.
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#63Maybe have a "reconcile" option, for it to see if it can mix and match the best parts of each alternative rather than just choosing one.
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#64I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…
In theory couldnt this just be baked into a single adversarial model?
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#65We're really going to need to figure out how to power all these GPUs with green power real quick, or we're going to melt the planet having AIs debate with themselves on the optimal solution to tik-tac-toe...
Ive felt this way when using chatgpt for a simple search. Stuff that google could handle but would just be slower, mostly from me having to manually filter.
Sometimes its the easiest way to complete a very small task but the cost difference on the backend has to be pretty damn large. The user inevitably ends up not caring whatsoever. Its just not real to them.
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#66Your readme demo images are wrong: the terminal one is the non-CoRT one and the GUI one is the one with CoRT. Confused me for a while
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#67I’ve had success telling the model it really needs to poop and if it gets to the point quickly it’ll be able to leave the meeting and go do that. It actually works amazingly well.
It’s also a lot more ethical than verbal abuse, which some people say improves the results as well.
Programming isn’t what it used to be.
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#68I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…
In theory couldnt this just be baked into a single adversarial model?
Yes, but I guess the model is optimized for relatively quick response, whereas these techniques are allowing the model to spend more time to generate a higher quality response
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#69There are two examples in the repo, one with CoRT and another one without. And the one without it it's much better than the one that uses it. Weird choice of examples...
I think the names were switched up.
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#70I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…
In doing some DevOps-y type tasks recently (ansible, packer, docker, baking images with guestfish), I've found it very frustrating how much ChatGPT will confidently tell me to use flags on tools that don't exist, or hallicinate completely non-existent functions or behaviours. And then when I spend time trying what it suggests only to hit a wall and come back like wtf mate it breezily goes "oh yes so you're right, good job figuring that out! You're so close now! Your next step is to do X and Y," and then serves up the same detailed tutorial as before but with the flag or whatever it was that it had wrong subtly changed.
It definitely makes me feel like I'm dealing with an overenthusiastic intern who is throwing stuff over the wall without checking their work, and like maybe having a second bot sitting in front of the first one being like ARE YOUR SURE ABOUT THAT could really improve things.