Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DiffusionGemma: 4x Faster Text Generation

blog.google

Post reply on HN
Print view View original

51–60 of 94 posts

Re: DiffusionGemma: 4x Faster Text Generation

#51

Post by epiccoleman »

onlyrealcuzzo wrote: post #49
bee_rider wrote: post #46

Earlier quoted context omitted.

How do you measure “true” complexity? Cyclomatic seems a bit… I dunno, artificial? Blunt? But it has the benefit of being defined.

There's a ton of research on this in the 80s... and interestingly, I haven't seen a lot of recent research. Surprisingly, it seems most languages don't have a standard package to do a lot of these detections. Ruby has Flay to detect similarity (something LLMs are prone to do). Basically re-write a huge function with only a couple of minor differences that should probably be params... One of the things I rely on most…

> Dynamically typed languages like Ruby create a huge surface area for type slop for LLMs, and why I would not recommend using a dynamically typed language for vibe coding.

I totally understand this, and have seen the problems firsthand. But Elixir / Phoenix / LiveView, along with Tidewave, have become my favorite "vibe slop stack." Just so quick and easy, and the LLM seems to get things right quite often.

Re: DiffusionGemma: 4x Faster Text Generation

#54

Post by nullc »

Has anyone evaluated any diffusion LLMs for error spotting?

E.g. run your normal autoregressive LLMs (with MTP whatever, as you like), then run a single diffusion pass over the result, and observe any tokens that diffusion thinks are unlikely.

Then prompt the autoregressive llm with some structured reasoning "Is an error? .."

Because the diffusion model is so structurally different perhaps it makes different errors such that this would provide gains even vs running distinct autoregressive LLMs which often make the same errors.

The same argument could apply for RWKV but it would be relatively expensive to apply it as a second pass on a big block of output, while it seems like a diffusion model would be cheaper.

Re: DiffusionGemma: 4x Faster Text Generation

#55

Post by desireco42 »

vineyardmike wrote: post #9

Recently I had switched to OpenCode to try out many of the Non-US-Frontier-Labs models. My unexpected favorite model to use was Mercury (a diffusion model). Not because it was “smart” but because it was stupid fast. It was more of a pair-programming experience instead of the SOTA agentic experience of prompting and waiting. Honestly, it was also way more fun and brought back some of the pre-AI coding experience while…

Wow... I forgot about that. Mercury is brutal. I had him review lint errors and the speed is just insane

Re: DiffusionGemma: 4x Faster Text Generation

#56

Post by petercooper »

I'm not getting anywhere near the speeds advertised on my 3090 Ti, alas, but it's fun watching it "fill out" its answers. I did Simon's "SVG pelican on a bicycle" test on it and the result was quite minimalistic but fit the brief: https://gist.github.com/peterc/7672e74ec1437945e5fca5ce2c1c9... -- this was on the Q4 quant running on patched llama.cpp. I will be interested to see if Simon's looks much different.

Re: DiffusionGemma: 4x Faster Text Generation

#57

Post by BarakWidawsky »

samuelknight wrote: post #21

Some of these comments miss the advantage of diffusion. This is will have a big impact on edge devices, such as your phone or the GPU in your computer. An LLM's decoder computes tokens one-at-a-time because attention has to account for each previous token. The existing LLM decoders scale well when you have enough load to batch many inferences together. Diffusion of limited benefit there. On edge you have a different…

You’re mostly right but conflating attention with autoregressive/causal which is the real issue that prevents you from using more compute

You can use diffusion with attention, and this model does in fact use attention

Re: DiffusionGemma: 4x Faster Text Generation

#60

Post by SwellJoe »

Google keeps flexin'. It's surprising that Gemini isn't more competitive against Claude or OpenAI models for code and agentic use, because it's clear Google still has some of the best AI people in the business. But, I guess Google is focused on stuff that runs on phones and near-realtime use cases, rather than the big thinky LLMs.

All these efficiency improvements seem likely to be really important to the future of AI, though, as the money starts flowing the other direction. The days of subsidized tokens to try to lock people into specific ecosystems are coming to an end, and we're going to have to start paying what it actually costs.

The companies that figure out how to make it cost-effective to run really smart models are the ones that will win. DeepSeek costs an order of magnitude less than GPT 5.5 or Opus 4.8. It's worse than either, but not catastrophically worse. I'll happily pay ten times as much for the best coding model, because it saves enough human time to justify it, but not a hundred times as much, which is where things seem to be heading (GPT 5.5 Pro cost over 200 times as much as DeepSeek in some benchmarks I recently did, and ~30 times as much as Opus 4.8).

Post reply on HN