Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multimodal Diffusion Language Models for Thinking-Aware Editing and Generation

github.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Multimodal Diffusion Language Models for Thinking-Aware Editing and Generation

#11

Post by lossolo »

NitpickLawyer wrote: post #7
soulofmischief wrote: post #6

Earlier quoted context omitted.

Diffusion text models aren't new, I've made them at home. Also, plenty of frontier models are good at tool calling, GPT-5 has just been trained to do it more so that it appears to do better at coding exercises with codex/IDEs. If you haven't tried an agentic IDE such as Cursor yet, or at least an extension such as Copilot, I would recommend checking them out and trying out Anthropic's models as well.

Do you have any examples / papers where they do the parallel thing proposed here? I've tried googles diffusion coding model, but AFAICT they don't do parallel thinking & coding. It seems to just take a prompt and output code. What's cool with this thinking & generation in parallel is that one can attend to the other. So you're not limited by prompt influences code, but can do prompt influences both thinking and code,…

They use bidirectional attention between modalities, not within the same modality. This doesn’t change much in the context you're referring to (coding). How do you think "thinking" works in current SOTA models like GPT-5-Thinking/Pro? When generating code, the model's "thinking" already attends to the code, and both influence each other during generation. "Reasoning" models modify the code as they generate it, they delete it, revise it, and adjust their internal reasoning based on the new tokens they produce during the "thinking" process. There are dozens of denoising models created for text, they are not good at it and parallel sampling between modalities will not change that.

Re: Multimodal Diffusion Language Models for Thinking-Aware Editing and Generation

#13

Post by Lerc »

jasonjmcghee wrote: post #10

Out of curiosity, is it possible this suffers from the same issues Anthropic found where reasoning expressed by the model and actual internal reasoning differ?

I think this is likely to happen in all models since their internal reasoning is not in the same form as the output. This is probably true also for humans.

This may solve the additional clouding that comes from LLMs using what is an effectively an iteration of instants to introspect the past. You cannot ask a autoregressive model what the thinking was behind the output because the only memory it has of the past is the output. It has to infer what it meant just the same as anyone else would.

To some extent this probably also happens in humans. You have richer memories, but you still do a lot of post hoc rationalisation.

Re: Multimodal Diffusion Language Models for Thinking-Aware Editing and Generation

#14

Post by observationist »

Lerc wrote: post #13
jasonjmcghee wrote: post #10

Out of curiosity, is it possible this suffers from the same issues Anthropic found where reasoning expressed by the model and actual internal reasoning differ?

I think this is likely to happen in all models since their internal reasoning is not in the same form as the output. This is probably true also for humans. This may solve the additional clouding that comes from LLMs using what is an effectively an iteration of instants to introspect the past. You cannot ask a autoregressive model what the thinking was behind the output because the only memory it has of the past is th…

Native latent reasoning, with latent aware RL scaffolding and all the rest will have to be built. If you use the direct text framework, you get confabulation / hallucination issues from the divergence between the tokens in the context and the rich activation representation that resulted in the output.

There are all sorts of places where the text and output is at least one degree of separation from the underlying activation vectors or other representations handled by a model, from floating point precision all the way up to tokenization abstraction, and a lot of experiments get run as if the tokens and context and representations are all one unified data concept. Have to match data abstractions appropriately, or the weird edge cases will break things in unexpected ways.

Re: Multimodal Diffusion Language Models for Thinking-Aware Editing and Generation

#15

Post by ricardobeat »

lossolo wrote: post #11
NitpickLawyer wrote: post #7

Earlier quoted context omitted.

Do you have any examples / papers where they do the parallel thing proposed here? I've tried googles diffusion coding model, but AFAICT they don't do parallel thinking & coding. It seems to just take a prompt and output code. What's cool with this thinking & generation in parallel is that one can attend to the other. So you're not limited by prompt influences code, but can do prompt influences both thinking and code,…

They use bidirectional attention between modalities, not within the same modality. This doesn’t change much in the context you're referring to (coding). How do you think "thinking" works in current SOTA models like GPT-5-Thinking/Pro? When generating code, the model's "thinking" already attends to the code, and both influence each other during generation. "Reasoning" models modify the code as they generate it, they d…

They cannot "edit" the code though, like you can with diffusion. They must emit all tokens again, or a patch/diff which is not directly connected to the previous stream of tokens.

Re: Multimodal Diffusion Language Models for Thinking-Aware Editing and Generation

#17

Post by lossolo »

ricardobeat wrote: post #15
lossolo wrote: post #11

Earlier quoted context omitted.

They use bidirectional attention between modalities, not within the same modality. This doesn’t change much in the context you're referring to (coding). How do you think "thinking" works in current SOTA models like GPT-5-Thinking/Pro? When generating code, the model's "thinking" already attends to the code, and both influence each other during generation. "Reasoning" models modify the code as they generate it, they d…

They cannot "edit" the code though, like you can with diffusion. They must emit all tokens again, or a patch/diff which is not directly connected to the previous stream of tokens.

LLMs can "edit" code, but as you say, they do it differently from diffusion models. They operate directly on long text sequences and use much more context, which is one reason they currently work better for coding. Diffusion models for code aren't a new idea, people have tried different designs, but so far they tend to underperform autoregressive LLMs, probably because denoising over discrete tokens is harder to make work than straightforward next token prediction.
Post reply on HN