Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini Diffusion

simonwillison.net

Post reply on HN
Print view View original

11–20 of 252 posts

Re: Gemini Diffusion

#11

Post by cubefox »

martincsweiss wrote: post #6
mountainriver wrote: post #5

Diffusion is more than just speed. Early benchmarks show it better at reasoning and planning pound for pound compared to AR. This is because it can edit and doesn’t suffer from early token bias.

This is a super interesting claim - can you point to these benchmarks?

https://deepmind.google/models/gemini-diffusion/#benchmarks

> Gemini Diffusion’s external benchmark performance is comparable to much larger models, whilst also being faster.

That doesn't necessarily mean that they scale as well as autoregressive models.

Re: Gemini Diffusion

#12

Post by nodja »

This is insanely fast, my guess is that the tradeoff here is that the GPUs will always be working at max capacity and there will be minimal compute savings from batching, which I realize now is not really a tradeoff.

My only worry is that the diffusion objective will be worse than AR in terms of model capabilities, if that's the case hopefully multi-token AR models will perform as well as diffusion, or we can use this as a draft model for speculative decoding.

Re: Gemini Diffusion

#13

Post by mdp2021 »

martincsweiss wrote: post #6
mountainriver wrote: post #5

Diffusion is more than just speed. Early benchmarks show it better at reasoning and planning pound for pound compared to AR. This is because it can edit and doesn’t suffer from early token bias.

This is a super interesting claim - can you point to these benchmarks?

Try this one:

# d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning

https://dllm-reasoning.github.io/

Re: Gemini Diffusion

#14

Post by ec109685 »

airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

If you make models fast enough, you can onboard that expert developer instantly and let them reason their way to a solution, especially when giving access to a RAG to.

Over time, I models will add more memory and institutional knowledge capture rather than starting from a blank slate each time.

Re: Gemini Diffusion

#16

Post by mdp2021 »

nodja wrote: post #12

This is insanely fast, my guess is that the tradeoff here is that the GPUs will always be working at max capacity and there will be minimal compute savings from batching, which I realize now is not really a tradeoff. My only worry is that the diffusion objective will be worse than AR in terms of model capabilities, if that's the case hopefully multi-token AR models will perform as well as diffusion, or we can use thi…

Why do you suspect dLLMs should not match (or surpass) arLLMs in quality? The general idea is that it is easier to treat the output as a structured whole (idea, points, concepts, words - in a tree) which is iteratively treated - that should go in the direction of "proper" quality.

Re: Gemini Diffusion

#17

Post by randomgoogler1 »

transformi wrote: post #2

Interesting to see if GROQ hardware can run this diffusion architecture..it will be two time magnitude of currently known speed :O

(Disc: Googler but don't have any specific knowledge of this architecture)

My understanding of Groq is that the reason it is fast is that all the weights are kept in SRAM and since the SRAM Compute bandwidth is much faster than HBM Compute bandwidth, you can generate tokens faster (During generation the main bottleneck is just bringing in the weights + KV caches into compute).

If the diffusion models just do multiple unmasked forward passes through a transformer, then the activation * weights computation + (attention computation) will be the bottleneck which will make each denoising step compute bound and there won't be any advantage in storing the weights in SRAM since you can overlap the HBM -> compute transfer with compute itself.

But my knowledge of diffusion is non-existent, so take this with a truck of salt.

Re: Gemini Diffusion

#18

Post by simonw »

synapsomorphy wrote: post #7

Nit: Diffusion isn't in place of transformers, it's in place of autoregression. Prior diffusion LLMs like Mercury [1] still use a transformer, but there's no causal masking, so the entire input is processed all at once and the output generation is obviously different. I very strongly suspect this is also using a transformer. [1] https://www.inceptionlabs.ai/introducing-mercury

Thanks, I updated my post to quote this comment.

Re: Gemini Diffusion

#20

Post by tripplyons »

cubefox wrote: post #9
synapsomorphy wrote: post #7

Nit: Diffusion isn't in place of transformers, it's in place of autoregression. Prior diffusion LLMs like Mercury [1] still use a transformer, but there's no causal masking, so the entire input is processed all at once and the output generation is obviously different. I very strongly suspect this is also using a transformer. [1] https://www.inceptionlabs.ai/introducing-mercury

Image diffusion models also use transformers nowadays. Here is the original "diffusion transformer" paper: https://arxiv.org/abs/2212.09748 Earlier image diffusion models used U-nets: https://en.wikipedia.org/wiki/U-Net

Many U-net based models such Stable Diffusion V1.5 modified the base architecture to include self-attention and cross-attention layers interleaved between convolution layers.
Post reply on HN