Consistency diffusion language models: Up to 14x faster, no quality loss
1–10 of 101 posts
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#2If this means there’s a 2x-7x speed up available to a scaled diffusion model like Inception Mercury, that’ll be a game changer. It feels 10x faster already…
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#3Google is working on a similar line of research. Wonder why they haven't rolled out a GPT40 scaled version of this yet
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#4Is anyone doing any form of diffusion language models that are actually practical to run today on the actual machine under my desk? There's loads of more "traditional" .gguf options (well, quants) that are practical even on shockingly weak hardware, and I've been seeing things that give me hope that diffusion is the next step forward, but so far it's all been early research prototypes.
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#5Is anyone doing any form of diffusion language models that are actually practical to run today on the actual machine under my desk? There's loads of more "traditional" .gguf options (well, quants) that are practical even on shockingly weak hardware, and I've been seeing things that give me hope that diffusion is the next step forward, but so far it's all been early research prototypes.
Based on my experience running diffusion image models I really hope this isn't going to take over anytime soon. Parallel decoding may be great if you have a nice parallel gpu or npu but is dog slow for cpus
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#6If this means there’s a 2x-7x speed up available to a scaled diffusion model like Inception Mercury, that’ll be a game changer. It feels 10x faster already…
Diffusion language models seem poised to smash purely autoregressive models. I'm giving it 1-2 years.
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#7Google is working on a similar line of research. Wonder why they haven't rolled out a GPT40 scaled version of this yet
Probably because it's expensive.
But I wish there were more "let's scale this thing to the skies" experiments from those who actually can afford to scale things to the skies.
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#8If this means there’s a 2x-7x speed up available to a scaled diffusion model like Inception Mercury, that’ll be a game changer. It feels 10x faster already…
Diffusion language models seem poised to smash purely autoregressive models. I'm giving it 1-2 years.
Feels like the sodium ion battery vs lithium ion battery thing, where there are theoretical benefits of one but the other has such a head start on commercialization that it'll take a long time to catch up.
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#9I wish there would be more of this research to speed things up rather than building ever larger models
Re: Consistency diffusion language models: Up to 14x faster, no quality loss
#10I wish there would be more of this research to speed things up rather than building ever larger models
Why not both?
Scaling laws are real! But they don't preclude faster processing.