I'm very interested in Diffusion text models. The concept of taking noise and adding words starting randomly all over the response, and filling in the noise from there on breaks my brain. I'm sure I have a fundamental misunderstanding of the technology, though.
Denoising is probably the weakest part of this model. There is recent research from Kaiming He showing that predicting the noise is actually not the best strategy for image generation since noise space is so large. Simply predicting the surface of the data you are trying to generate is far more representationally efficient, and perhaps in the next few months we'll see a version of that for LLMs
Diffusion language models work with a discrete output space, unlike image models that repeatedly refine a continuous output, so they don't do the noise-prediction thing anyway.