41–50 of 74 posts
This seems to be a system to generate better prompts to be fed into a base multimodal model. Interesting, but title is definitely clickbait.
Earlier quoted context omitted.
> I think is worth keeping in mind on the stack of possible approaches for, say agentic coding, that you can use a critic to not just ‘improve’ generated output, but most likely do some guided search through output space. The one issue I keep finding with those approaches is that there’s already good tools for the problem, but we keep searching for wasteful approaches because “natural languages” for something humans…
I hear that. Counterpoint - if you all you have is a Philips-head screwdriver, all you have is a Philips-head screwdriver. On the other hand if all you have is a six axis CnC mill, well, then you have a lot. I think of this less as audit misses, and more as developing a permanently useful tool. For open model weights, humanity will not (unless we’re talking real zombie apocalypse scenarios) lose these weights. They a…
I’ve read the paper and the skeptical comments here, to wit: it’s just an actor/critic pipeline by another name. I’ll bite and say this is actually interesting — and the paper title is misleading. What they’ve done here is hooked up a text-only LLM to multimodal critics, given it (mostly) an image diffusion generation task, and asked it to improve its prompting of the multimodal generation by getting a set of scores…
> zero shot I really wish we would find a different term for this. Doing something always takes at least one attempt, i.e. "one shotting". "Zero shotting" is an oxymoron, which makes it a term that only creates more confusion rather than succinctly conveying something.
I’ve read the paper and the skeptical comments here, to wit: it’s just an actor/critic pipeline by another name. I’ll bite and say this is actually interesting — and the paper title is misleading. What they’ve done here is hooked up a text-only LLM to multimodal critics, given it (mostly) an image diffusion generation task, and asked it to improve its prompting of the multimodal generation by getting a set of scores…
> zero shot I really wish we would find a different term for this. Doing something always takes at least one attempt, i.e. "one shotting". "Zero shotting" is an oxymoron, which makes it a term that only creates more confusion rather than succinctly conveying something.
I’ve read the paper and the skeptical comments here, to wit: it’s just an actor/critic pipeline by another name. I’ll bite and say this is actually interesting — and the paper title is misleading. What they’ve done here is hooked up a text-only LLM to multimodal critics, given it (mostly) an image diffusion generation task, and asked it to improve its prompting of the multimodal generation by getting a set of scores…
> zero shot I really wish we would find a different term for this. Doing something always takes at least one attempt, i.e. "one shotting". "Zero shotting" is an oxymoron, which makes it a term that only creates more confusion rather than succinctly conveying something.
TCGs have a related "zero turn win" concept, where the opponent goes first and you win without getting a turn due to the set of cards you randomly drew and being able to activate them on the opponent's turn.
Earlier quoted context omitted.
I hear that. Counterpoint - if you all you have is a Philips-head screwdriver, all you have is a Philips-head screwdriver. On the other hand if all you have is a six axis CnC mill, well, then you have a lot. I think of this less as audit misses, and more as developing a permanently useful tool. For open model weights, humanity will not (unless we’re talking real zombie apocalypse scenarios) lose these weights. They a…
While it is hard to argue with the wisdom of crystallizing intellectual capital into our tools, I do wonder if these models might be as likely to diminish as to develop the person using them, in which case we trade an implement's iterative improvement for ours, in a way
This was accurate. But mostly humans gained from books. I think we will develop the social technology to use these tools over time; giving some things up and gaining others.
If we don’t, the Amish can just take over and be like “Stupid English, using the devil’s weights.” :)
I’ve read the paper and the skeptical comments here, to wit: it’s just an actor/critic pipeline by another name. I’ll bite and say this is actually interesting — and the paper title is misleading. What they’ve done here is hooked up a text-only LLM to multimodal critics, given it (mostly) an image diffusion generation task, and asked it to improve its prompting of the multimodal generation by getting a set of scores…
> zero shot I really wish we would find a different term for this. Doing something always takes at least one attempt, i.e. "one shotting". "Zero shotting" is an oxymoron, which makes it a term that only creates more confusion rather than succinctly conveying something.
Earlier quoted context omitted.
Are they using the same diffusion models as the GPT-3 area? Meaning is it the LLM that has improved or is it the diffusion model? I know it's probably a foolish take but I am really skeptical of the "larger models will solve all our problems" line of thinking.
They don’t compare in the paper. I will say I experimented extensively with GPT-3 era LLMs on improving ouput by trying to guide early diffusion models with critical responses. It was a) not successful, and b) pretty clear to me that GPT-3 didn’t “get” what it was supposed to be doing, or didn’t have enough context to keep all this in mind, or couldn’t process it properly, or some such thing. This paper has ablations…
100% agree. I'd classify the time now as identifying the limits of what they can functionally do though, an it's a lot!