Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introspective Diffusion Language Models

introspective-diffusion.github.io

Post reply on HN
Print view View original

21–30 of 60 posts

Re: Introspective Diffusion Language Models

#21

Post by Topfi »

andsoitis wrote: post #2

Is anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!

I've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it.

Sadly, it does not perform at the level of e.g. Haiku 3.5 for tool calling, despite their own benchmarks claiming parity with Haiku 4.5, but it does compete with Flash Lite there too.

Anything with very targeted output, sufficient existing input and that benefits from a seamless feeling lends itself to dLLMs. Could see a place in tab-complete too, though Cursors model seems to be sufficiently low latency already.

Re: Introspective Diffusion Language Models

#22

Post by qeternity »

awestroke wrote: post #17
thepasch wrote: post #9

If I’m reading this right, this is pretty wild. They turned a Qwen autoregressor into a diffuser by using a bunch of really clever techniques, and they vastly outperform any “native diffuser,” actually being competitive with the base model they were trained from. The obvious upside here is the massive speedup in generation. And then through a LoRA adapter, you can ground the diffuser on the base model’s distribution…

I don't understand how you can compare against the base model output without generating with the base model, in which case what's the point?

I haven't read TFA yet but a common technique is speculative decoding where a fast draft model will generate X tokens, which are then verified by the larger target model. The target model may accept some Y So let's say a draft model generates 5 tokens, all 5 of these can be verified in parallel with a single forward pass of the target model. The target model may only accept the first 4 tokens (or whatever) but as long as the 5 forward passes of the draft model + 1 prefill of the target model is faster than 4 forward passes of the target, you will have a speedup while maintaining the exact output distribution as the target.

Re: Introspective Diffusion Language Models

#23

Post by anentropic »

awestroke wrote: post #17
thepasch wrote: post #9

If I’m reading this right, this is pretty wild. They turned a Qwen autoregressor into a diffuser by using a bunch of really clever techniques, and they vastly outperform any “native diffuser,” actually being competitive with the base model they were trained from. The obvious upside here is the massive speedup in generation. And then through a LoRA adapter, you can ground the diffuser on the base model’s distribution…

I don't understand how you can compare against the base model output without generating with the base model, in which case what's the point?

presumably that happens at training time?

then once successfully trained you get faster inference from just the diffusion model

Re: Introspective Diffusion Language Models

#25

Post by Topfi »

simianwords wrote: post #16
moeadham wrote: post #15

Earlier quoted context omitted.

Well, you can take the output of a first pass and pass it back through the model like AR “reasoning” models do at inference time.

Yes and has this been tried?

Yes, Mercury 2 is a reasoning model [0].

[0] https://docs.inceptionlabs.ai/get-started/models#mercury-2

Re: Introspective Diffusion Language Models

#26

Post by nl »

Topfi wrote: post #21
andsoitis wrote: post #2

Is anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!

I've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it. Sadly, it does not perform at the level o…

If you like Mercury 2 you should try Xiaomi Mimo-v2-flash.

I have an agentic benchmark and it shows Mercury 2 at 19/25 in 58 seconds and Mimo v2 Flash at 22/25 in 109 seconds

https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo... (flip to the Cost vs Performance tab to see speed more graphically too)

Re: Introspective Diffusion Language Models

#27

Post by nl »

IanCal wrote: post #6
recsv-heredoc wrote: post #4

Earlier quoted context omitted.

https://www.inceptionlabs.ai/ This startup seems to have been at it a while. From our look into it - amazing speed, but challenges remain around time-to-first-token user experience and overall answer quality. Can absolutely see this working if we can get the speed and accuracy up to that “good enough” position for cheaper models - or non-user facing async work. One other question I’ve had is wondering if it’s possibl…

The overall speed rather than TTFT might start to be more relevant as the caller moves from being a human to another model. However quality is really important. I tried that site and clicked one of their examples, "create a javascript animation". Fast response, but while it starts like this ``` Below is a self‑contained HTML + CSS + JavaScript example that creates a simple, smooth animation: a colorful ball bounces a…

Mercury 2 is better than that in my testing, but it does have trouble with tool calling.

Re: Introspective Diffusion Language Models

#28

Post by feznyng »

Topfi wrote: post #21
andsoitis wrote: post #2

Is anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!

I've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it. Sadly, it does not perform at the level o…

Did you get a chance to evaluate coding performance?

Re: Introspective Diffusion Language Models

#29

Post by Balinares »

awestroke wrote: post #17
thepasch wrote: post #9

If I’m reading this right, this is pretty wild. They turned a Qwen autoregressor into a diffuser by using a bunch of really clever techniques, and they vastly outperform any “native diffuser,” actually being competitive with the base model they were trained from. The obvious upside here is the massive speedup in generation. And then through a LoRA adapter, you can ground the diffuser on the base model’s distribution…

I don't understand how you can compare against the base model output without generating with the base model, in which case what's the point?

Isn't that exactly how draft models speed up inference, though? Validating a batch of tokens is significantly faster than generating them.

Re: Introspective Diffusion Language Models

#30

Post by Topfi »

nl wrote: post #26
Topfi wrote: post #21

Earlier quoted context omitted.

I've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it. Sadly, it does not perform at the level o…

If you like Mercury 2 you should try Xiaomi Mimo-v2-flash. I have an agentic benchmark and it shows Mercury 2 at 19/25 in 58 seconds and Mimo v2 Flash at 22/25 in 109 seconds https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo... (flip to the Cost vs Performance tab to see speed more graphically too)

Thanks for the recommendation and sharing your evals, will take a closer look at them. Yes, the Mimo models are very interesting, end-to-end pricing wise especially, though in my tool call runs, GLM 4.7 Flash did slightly better at roughly equal speed and full run cost. Is of course very task dependent and both are amazing options in the price range, but latency wise, nothing feels like Mercury 2 at the moment.
Post reply on HN