Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini Diffusion

simonwillison.net

Post reply on HN
Print view View original

181–190 of 252 posts

Re: Gemini Diffusion

#181

Post by simonw »

beernet wrote: post #144

Serious question: Why does it appear that pages from this URL very often end up on top of HN? I don't find the content particularly special compared to the average HN post. Does the algorithm prefer certain URLs?

If anything HN is getting harder for me to get stuff on these days. Most of my self submissions of my content have failed to chart over the past few months: https://news.ycombinator.com/submitted?id=simonw

You can see submissions from my domain by other people here: https://news.ycombinator.com/from?site=simonwillison.net

It's weird what DOES make it. I had high hopes for my piece on ChatGPT memory yesterday - https://simonwillison.net/2025/May/21/chatgpt-new-memory/ - and it got nowhere. This Gemini piece was much more of a throwaway note, I mainly wanted to mark the release of an influential new model on my own site.

The reason I get content on here more than most people is that I write a LOT more than most people. So far this year I've published 45 long form blog entries and 274 short form (link blog) entries - this Gemini piece is one of those.

I try to always add something new - for this Gemini piece that was the video of it running, the references back to a similar demo from Cerebras and (updated since I first posted) a couple of quotes Hacker News comments to help explain diffusion LLMs better.

I wrote more about my approach to blogging here: https://simonwillison.net/2024/Dec/22/link-blog/

The sad truth is that very few people produce long-form writing online these days! Most people who publish regularly are doing tweets, LinkedIn posts and short-form video instead.

Re: Gemini Diffusion

#182

Post by behnamoh »

tok/s speeds in the video:

- 1st message (empty context): 857 tok/s

- 2nd message (2244 tokens in context): 727 tok/s

- 3rd message (2244+1398 tokens in context): 693 tok/s

I'm no expert in diffusion models but this looks like a drastic drop in speed, especially in longer chats (this was just 3 messages).

Re: Gemini Diffusion

#183

Post by jgalt212 »

loudmax wrote: post #155
jgalt212 wrote: post #151

Is this more evidence that the days of insatiable demand for GPU data centers and electricity are behind us?

If the diffusion models are an improvement over autoregression models, then the answer is No, due to Jevon's paradox. That is, as these models get cheaper and better, they provide more utility, driving more demand. Even as your datacenters become more productive, the demand for their compute power increases at an even faster pace. The thing that will limit demand for compute is when the world decides it has sufficien…

Perhaps, but it's not clear that Jevon's paradox is at play here. AI / LLM uptake has been muted, or lacking legs, outside of coding. And it's not because of cost (AI inference is being provided below cost by megatech).

Re: Gemini Diffusion

#184

Post by angusturner »

skydhash wrote: post #163
nvtop wrote: post #152

Earlier quoted context omitted.

Correct, diffusion LMs can edit their intermediate predictions, so "final" tokens aren't necessarily final. This is an exciting property because it allows models to correct errors in what's generated so far -- something that GPT-like models can't. This editing is based on the Transformer's encoder property to predict token probabilities for __every__ token in a sequence, not just for [MASK]s. So when you input a sent…

But what about the dependency graph between symbols in the program. Because all those symbols have high constraints around them which is the program design. The issue comes in image diffusion as well. When you ask it for a portrait and some details are wrong. That’s because the face has constraints (which you learn about as an artist). Patterns and probability won’t help you.

You assume that for small steps (I.e taking some noisy code and slightly denoising) you can make an independence assumption. (All tokens conditionally independent, given the current state).

Once you chain many steps you get a very flexible distribution that can model all the interdependencies.

A stats person could probably provide more nuance, although two interesting connection I’ve seen: There is some sense in which diffusion generalises autoregression, because you don’t have to pick an ordering when you factor the dependency graph.

(Or put otherwise, for some definitions of diffusion you can show autoregression to be a special case).

Re: Gemini Diffusion

#185

Post by angusturner »

One under appreciated / misunderstood aspect of these models is they use more compute than an equivalent sized autoregressive model.

It’s just that for N tokens, autoregressive model has to make N sequential steps.

Where diffusion does K x N, with the N being done in parallel. And for K This makes me wonder how well they will scale to many users, since batching requests would presumably saturate the accelerators much faster?

Although I guess it depends on the exact usage patterns.

Anyway, very cool demo nonetheless.

Re: Gemini Diffusion

#187

Post by Onawa »

westoncb wrote: post #123
airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

The trick to this is you've got to talk to them and share this information in the same way. I can give an example. These days my main workflow is as follows: if I have some big feature/refactor/whatever I'm going to work on I'll just start talking to o3 about it essentially as if it was a coworker and (somewhat painstakingly) paste in relevant source files it needs for context. We'll have a high-level discussion abou…

I would suggest trying the Continue.dev VSCode plugin for selective context injection. The plugin is Apache 2.0 licensed, and you can hook it up to any LLM API including local.

It has most of the same features as GitHub Copilot, but a few extra features I find essential. It can scrape documentation sites for individual libraries, which means you can do stuff like `@pandas @terminal @codebase Help me fix this error`.

For greenfield projects I will usually start out in a web-based chat interface, but the second I need to go back and forth between IDE and the web I switch over to the Continue.dev plugin.

Re: Gemini Diffusion

#188

Post by petercooper »

Jackson__ wrote: post #83

What is this blog spam doing here? This is has literally no new information compared to the official release page. It would make a lot more sense to change the link to https://deepmind.google/models/gemini-diffusion/ to discuss the topic.

TBH I learnt more from Simon's post than I did actually being physically on site at I/O where it was barely covered at all.

Re: Gemini Diffusion

#189

Post by westoncb »

Onawa wrote: post #187
westoncb wrote: post #123

Earlier quoted context omitted.

The trick to this is you've got to talk to them and share this information in the same way. I can give an example. These days my main workflow is as follows: if I have some big feature/refactor/whatever I'm going to work on I'll just start talking to o3 about it essentially as if it was a coworker and (somewhat painstakingly) paste in relevant source files it needs for context. We'll have a high-level discussion abou…

I would suggest trying the Continue.dev VSCode plugin for selective context injection. The plugin is Apache 2.0 licensed, and you can hook it up to any LLM API including local. It has most of the same features as GitHub Copilot, but a few extra features I find essential. It can scrape documentation sites for individual libraries, which means you can do stuff like `@pandas @terminal @codebase Help me fix this error`.…

I’m pretty happy with Zed for development. I do plan on developing custom tooling around my style of workflow, but it’s not going to be part of an IDE.

Re: Gemini Diffusion

#190

Post by aposm »

airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

A human working on an existing codebase does not have any special signal about what is _not_ in a codebase. Instead, a (good) human engineer can look at how a problem is handled and consider why it might have been done that way vs other options, then make an educated decision about whether that alternative would be an improvement. To me this seems like yet another piece of evidence that these models are not doing any "reasoning" or problem-solving.
Post reply on HN