[dead]
Introspective Diffusion Language Models
11–20 of 60 posts
Re: Introspective Diffusion Language Models
#12> 2025-04-12: Initial code release with training and inference support.
> 2025-04-12: Released I-DLM-8B, I-DLM-32B, and I-DLM-8B-LoRA on HuggingFace.
Is this old already? Not saying that's a bad thing, since it seems very sophisticated. Just curious if there's an update
Re: Introspective Diffusion Language Models
#13Can diffusion models have reasoning steps where they generate a block, introspect and then generate another until the output is satisfactory?
Re: Introspective Diffusion Language Models
#14> 2025-04-12: Initial code release with training and inference support. > 2025-04-12: Released I-DLM-8B, I-DLM-32B, and I-DLM-8B-LoRA on HuggingFace. Is this old already? Not saying that's a bad thing, since it seems very sophisticated. Just curious if there's an update
It's clearly a typo on the year, April 12 was two days ago, a quick check in HuggingFace shows that they were uploaded 5 days ago.
Re: Introspective Diffusion Language Models
#15Can diffusion models have reasoning steps where they generate a block, introspect and then generate another until the output is satisfactory?
Well, you can take the output of a first pass and pass it back through the model like AR “reasoning” models do at inference time.
Re: Introspective Diffusion Language Models
#16Can diffusion models have reasoning steps where they generate a block, introspect and then generate another until the output is satisfactory?
Well, you can take the output of a first pass and pass it back through the model like AR “reasoning” models do at inference time.
Yes and has this been tried?
Re: Introspective Diffusion Language Models
#17If I’m reading this right, this is pretty wild. They turned a Qwen autoregressor into a diffuser by using a bunch of really clever techniques, and they vastly outperform any “native diffuser,” actually being competitive with the base model they were trained from. The obvious upside here is the massive speedup in generation. And then through a LoRA adapter, you can ground the diffuser on the base model’s distribution…
I don't understand how you can compare against the base model output without generating with the base model, in which case what's the point?
Re: Introspective Diffusion Language Models
#18Is anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!
It's being explored right now for speculative decoding in the local-LLM space, which I think is quite interesting as a use-case https://www.emergentmind.com/topics/dflash-block-diffusion-f...
DFlash immediately came to my mind.
There are several Mac implementations of it that show > 2x faster Qwen3.5 already.
Re: Introspective Diffusion Language Models
#19If I’m reading this right, this is pretty wild. They turned a Qwen autoregressor into a diffuser by using a bunch of really clever techniques, and they vastly outperform any “native diffuser,” actually being competitive with the base model they were trained from. The obvious upside here is the massive speedup in generation. And then through a LoRA adapter, you can ground the diffuser on the base model’s distribution…
I don't understand how you can compare against the base model output without generating with the base model, in which case what's the point?
You would only use the base model during training. This is a distillation technique
Re: Introspective Diffusion Language Models
#20If I’m reading this right, this is pretty wild. They turned a Qwen autoregressor into a diffuser by using a bunch of really clever techniques, and they vastly outperform any “native diffuser,” actually being competitive with the base model they were trained from. The obvious upside here is the massive speedup in generation. And then through a LoRA adapter, you can ground the diffuser on the base model’s distribution…
I think your excitement is justified. The paper is claiming a serious bridge between AR quality and parallel decoding, and the lossless LoRA-assisted mode is the wildest part.