Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

github.com

Post reply on HN
Print view View original

91–100 of 253 posts

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#91

Post by vunderba »

Lerc wrote: post #54

I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…

A year or so ago I experimented with splitting a user prompt down to a set of "different AI personas" that would each try to approach the user's problem in a different way and then bubble back up with a master arbiter for consensus.

I modeled it after the concept of advisors from Civilization II. It worked reasonably well though I think it was at least somewhat limited by being constrained to a single LLM (Mistral). It also lit my computer on fire.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#92

Post by vunderba »

mikepurvis wrote: post #70
Lerc wrote: post #54

I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…

In doing some DevOps-y type tasks recently (ansible, packer, docker, baking images with guestfish), I've found it very frustrating how much ChatGPT will confidently tell me to use flags on tools that don't exist, or hallicinate completely non-existent functions or behaviours. And then when I spend time trying what it suggests only to hit a wall and come back like wtf mate it breezily goes "oh yes so you're right, goo…

100%. This has happened enough to me that I wished I could just inject the man page docs into it to at least act as a sanity check.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#93

Post by tinix »

noworriesnate wrote: post #67

I’ve had success telling the model it really needs to poop and if it gets to the point quickly it’ll be able to leave the meeting and go do that. It actually works amazingly well. It’s also a lot more ethical than verbal abuse, which some people say improves the results as well. Programming isn’t what it used to be.

this works for getting out of traffic tickets too lol

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#94

Post by danielmarkbruce »

Lerc wrote: post #54

I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…

This is being done, and you could apply it to a lot of domains. Go for it for whatever use case you have.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#95

Post by 0x20cowboy »

mikepurvis wrote: post #70
Lerc wrote: post #54

I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the t…

In doing some DevOps-y type tasks recently (ansible, packer, docker, baking images with guestfish), I've found it very frustrating how much ChatGPT will confidently tell me to use flags on tools that don't exist, or hallicinate completely non-existent functions or behaviours. And then when I spend time trying what it suggests only to hit a wall and come back like wtf mate it breezily goes "oh yes so you're right, goo…

I did a stint in Devops and I found every models to be like this for all of the infra-as-code languages. Anything yaml based was especially bad.

Even Amazon’s own offering completely made things up about Amazon’s own formats.

I’d be curious as to why that is. It seems like there would be enough training data, and for Amazon in particular it seems like they could make a validation tool the model could use.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#96

Post by hsuduebc2 »

odo1242 wrote: post #88

Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification…

We're there any situation that first conclusion from AI was completely changed? Can you give generally examples of situations where it changed or significantly improved overall result? It sounds cool.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#97

Post by JumpCrisscross »

odo1242 wrote: post #88

Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification…

Kagi’s Assistant feature makes this super easy. Just switch assistants and ask them to check the other’s work.

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#98

Post by inanutshellus »

dingnuts wrote: post #59
jbm wrote: post #49

Earlier quoted context omitted.

I disagree. If anything, telling GPT to be blunt seems to downgrade its IQ; it hallucinates more and makes statements without considering priors or context. I jokingly call it Reddit mode.

why would that be a joke? there's a ton of Reddit comments in the training data, and the output is of similar quality. LLMs are literally outputting average Reddit comments.

See, he's not joking, he's "joking" ...

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#99

Post by ChadMoran »

Fast Agent has this as a first-class citizen called "Evaluator Optimizer" pattern. Where it in a loop with a defined number of max refinements judge itself and give the output a rating, demanding it improve it's output.

Highly encourage others to check out Fast Agent. It has been delightful to use. It has interactive chat mode which I love and it's really tight and easy to implement.

https://github.com/evalstate/fast-agent

Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

#100

Post by ChadMoran »

electroly wrote: post #22

This seems to be different than I expected from the title. I thought it would be explicitly adversarial. 1. You are the assistant. Please answer the question directly. 2. You are the cross-examiner. The assistant is wrong. Explain why. 3. You are the assistant. The cross-examiner is wrong. Defend your claim. 4. You are a judge. Did either party make their case, or is another round of argumentation required? I haven't…

Check out Fast Agent! (I have no affiliation with it, just use it).

https://github.com/evalstate/fast-agent

Post reply on HN