all this hard thinking yet humanity fails to come up with just one girlfriend for me
Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
181–190 of 253 posts
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#182Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification…
I've wondered if it might be helpful to randomly "shard" training data between two LLMs; just feed half the training data to one, and the rest to the other, with no overlap. So instead of using two models, you'd be making two halves of one model do a similar (deliberative) process to yours. I wonder if that would result in a benefit over a single model with the full training set, and if you could continue to do the s…
There's some precedent for that: you can do some useful things with the cross entropy of the two models. And k-fold cross validation might also be relevant.
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#183Earlier quoted context omitted.
Techniques like this have been around since GPT-3.5. There are boatloads of papers on the topic. I have no idea why anyone thinks this is novel. I guess that speaks to the state of HN
Exactly... I thought that implementing STORM was just a basic step in this topic... Looks like we're running in circles.
Mind sharing a link?
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#184Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification…
i dont understand, is it doing your schoolwork?
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#185Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification…
Ah, now we know why Spain was out of electricity yesterday.
Oh that was a good one XD
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#186Earlier quoted context omitted.
"conceptual understanding of something" is just another way of saying "the relationship between concepts", which is exactly what transformer models use. *EDIT* To elaborate, how can you define anything in isolation of every other concept/thing? You can't. Things are only defined by their relationships to each other, which is exactly the same thing transformer models do.
No, it isn't. "Conceptual understanding" is a deep comprehension of a particular concept. It is grasping its meaning, significance, applications, and boundaries. It involves knowing not just what something is definitionally, but understanding how it works, why it matters, and how it connects to other ideas. "The relationship between concepts," is focusing specifically on how different ideas connect, overlap, contradi…
> It is grasping its meaning, significance, applications, and boundaries
To define "thinking" by using words like "meaning", "understanding", or "comprehension" just moves the need for definition further up the abstraction ladder. It doesn't help to define what "thinking" is in any quantifiable way.
To play along, could you define "meaning" or "understanding" in a way that doesn't resort to ghost-talk or just move the definition even further up the abstraction ladder? They are both subjective terms that describe how humans feel, not well defined words that describe objective reality in some way.
To use a more quantifiable metric we could look at something like Humanity's Last Exam. OpenAI's o3 scores something like 20% (a feat which few humans could accomplish). To put that in perspective, consider that fifty four percent of Americans now read below the sixth grade level. Like it or not the machines are "smarter" than the majority of humans and have deeper "understanding" in most of the objective ways we've thought of to measure it. Subjective feelings aside, it's tough to argue that the machines aren't conscious if we're going to accept that our fellow citizens are.
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#187That is just reinforcement learning in disguise
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#188Adversarial networks have been a thing for a while
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#189Earlier quoted context omitted.
Have you tried n8n? It allows you to build flows like that - you can run the community version in a Docker container within a few minutes and share the configurations for the flows you have built very easily.
_#_ has to be one of the worst word shortening schemes I've ever seen get widespread. It only works with a very small number of long-lived technologies, in which case they basically just get a nickname, "k8s" "i18n". It does not at all work for larger contexts. You're basically making someone solve a crossword (2 across, 10 letters with two filled in) just to parse your sentence.
The app is actually called n8n - https://n8n.io/
Re: Chain of Recursive Thoughts: Make AI think harder by making it argue with itself
#190I see a lot of threads pitting models against each other (or whole swarms of them) in the hope that "wisdom of crowds" will magically appear. After a stack of experiments of my own—and after watching the recent ASU/Microsoft-Research work [1].. I've landed on a simpler takeaway:
An LLM is a terrible verifier of another LLM. Subbarao Kambhampati's "(How) Do LLMs Reason/Plan?" talk shows GPT-4 confidently producing provably wrong graph-coloring proofs until a symbolic SAT solver is introduced as the referee [1]. Stechly et al. quantify the problem: letting GPT-4 critique its own answers *reduces* accuracy, whereas adding an external, sound verifier boosts it by ~30 pp across planning and puzzle tasks [2]. In other words, verification is *harder* than generation for today's autoregressive models, so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.).
Because of that asymmetry, stacking multiple LLMs rarely helps. The "LLM-Modulo" position paper argues that auto-regressive models simply can't do self-verification or long-horizon planning on their own and should instead be treated as high-recall idea generators wrapped by a single, sound verifier [3]. In my tests, replacing a five-model "debate" with one strong model + verifier gives equal or better answers with far less latency and orchestration overhead.
[1] https://www.youtube.com/watch?v=0u2hdSpNS2o - (How) Do LLMs Reason/Plan? (talk at Microsoft Research, 11 Apr 2025)
[2] https://arxiv.org/abs/2402.08115
[3] https://arxiv.org/abs/2402.01817 (related to the talk in #1)