241–250 of 253 posts
I see a lot of threads pitting models against each other (or whole swarms of them) in the hope that "wisdom of crowds" will magically appear. After a stack of experiments of my own—and after watching the recent ASU/Microsoft-Research work [1].. I've landed on a simpler takeaway: An LLM is a terrible verifier of another LLM. Subbarao Kambhampati's "(How) Do LLMs Reason/Plan?" talk shows GPT-4 confidently producing pro…
Earlier quoted context omitted.
> ...so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.). Agree. What do you think about telling the LLM to also generate unit tests for the code it spits and then run all tests (including previous application unit tests). I think this is a way to ensure some level of grounded verification: - Does code compile? - Do unit test pass? AI can then consume…
This works well but only if you eyeball the tests and edit them a bit in my experience. Otherwise it gets lazy and makes them trivial to pass. Also, you’ve often gotta explicitly tell it not to hardcode test cases in the solution to make them pass.
You can use property based testing for that.
But I've often run into cases where the AI gets into a vicious spiral of worse and worse code when you keep feeding it the test failures.
Earlier quoted context omitted.
_#_ has to be one of the worst word shortening schemes I've ever seen get widespread. It only works with a very small number of long-lived technologies, in which case they basically just get a nickname, "k8s" "i18n". It does not at all work for larger contexts. You're basically making someone solve a crossword (2 across, 10 letters with two filled in) just to parse your sentence.
It's just another form of any other jargon - unknown until you know it, and usually specific to the use case. I see k8s and i18n or a11y and I know exactly what they mean because at some point I learned it and it's part of the world I live in. Searching for stuff is how we learn, not solving crosswords.
Earlier quoted context omitted.
To be honest, this is what I assumed this repo was doing from the title. It talks about arguing with itself, but it looks like it's just generating multiple alternative responses in parallel and selecting the best one. Do you find your method handles "sycophancy" well?
I don’t really know. I stopped using ChatGPT at some point because I disliked how cagey it became about a lot of topics. I used to enjoy making write improbable movies mashup when GPT3 was released and at some point it became very touchy about IP rights and violence which was annoying. I generally use Deepseek nowadays which is not sycophantic and surprisingly doesn’t seem as censored to me especially if you use a ve…
Earlier quoted context omitted.
LLMs learn high-dimensional representations that capture conceptual relationships in their training data. They manipulate those representations in ways that approximate human reasoning.
> They manipulate those representations in ways that approximate human reasoning. Fwiw, this is the story of my life. Seriously.
System 1 vs System 2 thinking.
System 1 is rapid, uses heuristics to make quick judgements. Not rigorous. System 1 is the default mode.
System 2 is slow deliberate reasoning, energy intensive, and even humans get that wrong.
LLMs often use something like System 1 pattern matching, get the answer wrong initially, then can be prodded into trying again with a System 2 approach (chain of thought).
I see a lot of threads pitting models against each other (or whole swarms of them) in the hope that "wisdom of crowds" will magically appear. After a stack of experiments of my own—and after watching the recent ASU/Microsoft-Research work [1].. I've landed on a simpler takeaway: An LLM is a terrible verifier of another LLM. Subbarao Kambhampati's "(How) Do LLMs Reason/Plan?" talk shows GPT-4 confidently producing pro…
Your references show me that it is absolutely task depended. In many domains it's true that "criticizing is easier than creating". The best example might be books and movies, where it's trivial to say the characters were shallow, but it's surprisingly hard to create deeply interesting characters. In Software Engineering, there are similar dynamics. An LLM with a security vuln finding prompt will be able to point out…
Earlier quoted context omitted.
Your references show me that it is absolutely task depended. In many domains it's true that "criticizing is easier than creating". The best example might be books and movies, where it's trivial to say the characters were shallow, but it's surprisingly hard to create deeply interesting characters. In Software Engineering, there are similar dynamics. An LLM with a security vuln finding prompt will be able to point out…
An LLM cannot reason through a mathematical proof, it would be something other than an LLM if it could.
Earlier quoted context omitted.
I believe, what the smart AI company is trying to do, right now, in secret, is to use US, the humans, and our replies to the AIs, as training for the next generation of self-verifying-models. :) Training on corpus data gets you to 1 order of magnitude. But training on interactive data where you can observe and adapt to the OODA-loop? So much more powerful. At least, that's what I'd be doing if I were doing AI :) But…
I think you'd need to screen for quality of response quite stringently as loads of people will produce "corrections" which are just plain wrong.
I see a lot of threads pitting models against each other (or whole swarms of them) in the hope that "wisdom of crowds" will magically appear. After a stack of experiments of my own—and after watching the recent ASU/Microsoft-Research work [1].. I've landed on a simpler takeaway: An LLM is a terrible verifier of another LLM. Subbarao Kambhampati's "(How) Do LLMs Reason/Plan?" talk shows GPT-4 confidently producing pro…
In the case described, the generator is an LLM, and the tester (called a "verifier") is "the compiler, linter, SAT solver, ground truth dataset, etc."
And of course generate-and-test is related to trial-and-error, which has probably existed since the Paleolithic.
We're really going to need to figure out how to power all these GPUs with green power real quick, or we're going to melt the planet having AIs debate with themselves on the optimal solution to tik-tac-toe...