Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini Diffusion

simonwillison.net

Post reply on HN
Print view View original

111–120 of 252 posts

Re: Gemini Diffusion

#111

Post by Incipient »

fragmede wrote: post #75
Incipient wrote: post #65

Earlier quoted context omitted.

I'm in the camp of 'no good for existing'. I try to get ~1000 line files refactored to use different libraries, design paradigms, etc and it usually outputs garbage - pulling db logic into the UI, grabbing unrelated api/function calls, to entirely just corrupting the output. I'm sure there is a way to correctly use this tool, so I'm feeling like I'm "just holding it wrong".

Which LLM are you using? what LLM tool are you using? What's your tech stack that you're generating code for? Without sharing anything you can't, what prompts are you using?

Was more of a general comment - I'm surprised there is significant variation between any of the frontier models?

However, vscode with various python frameworks/libraries; dash, fastapi, pandas, etc. Typically passing the 4-5 relevant files in as context.

Developing via docker so I haven't found a nice way for agents to work.

Re: Gemini Diffusion

#112

Post by FieryTransition »

ManuelKiessling wrote: post #81
airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

"...what is not in a codebase, and there is meaningful signal in that negative space." Man, I'm writing software for money for decades now, but this fundamental truth never occured to me, at least not consciously and with such clarity. So, thank you!

There's a reason why less is called less, and not more.

Re: Gemini Diffusion

#113

Post by [deleted] »

wenc wrote: post #74
efitz wrote: post #66

Earlier quoted context omitted.

No. Spell check frequently still gets things wrong if the word is spelled correctly and the sentence is grammatically correct but the wrong word was used.

Can you give me an example? Spell check only checks if a word is in dictionary. It doesn’t check grammar or context.

[deleted]

Re: Gemini Diffusion

#114

Post by danielbln »

Incipient wrote: post #111
fragmede wrote: post #75

Earlier quoted context omitted.

Which LLM are you using? what LLM tool are you using? What's your tech stack that you're generating code for? Without sharing anything you can't, what prompts are you using?

Was more of a general comment - I'm surprised there is significant variation between any of the frontier models? However, vscode with various python frameworks/libraries; dash, fastapi, pandas, etc. Typically passing the 4-5 relevant files in as context. Developing via docker so I haven't found a nice way for agents to work.

I would suggest using an agentic system like Cline, so that the LLM can wander through the codebase by itself and do research and build a "mental model" and then set up an implementation plan. The you iterate in that and hand it off for implementation. This flow works significantly better than what you're describing.

Re: Gemini Diffusion

#115

Post by spuz »

ManuelKiessling wrote: post #81
airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

"...what is not in a codebase, and there is meaningful signal in that negative space." Man, I'm writing software for money for decades now, but this fundamental truth never occured to me, at least not consciously and with such clarity. So, thank you!

I am not certain that I agree with this. If there are alternative ways of solving a problem that we're not taken then these should be documented in comments. A mantra I try to tell myself and my colleagues is if information exists in your brain and nowhere else then write down it down _somewhere_. If I tried 5 different libraries before settling on one, then I write in comments which libraries I tried but didn't work and why. If I used a particular tool to debug a race condition then I put a link to a wiki page on how to use it in the comments. If we have one particular colleague who is an expert in some area then I write their name in a comment. Basically anything that is going to save future developers' time should be written down.

Re: Gemini Diffusion

#116

Post by impossiblefork »

shreezus wrote: post #50

Is anyone else totally blown away by this? I feel like it’s easily the biggest announcement out of IO, however it’s been overshadowed by Veo 3 etc. Diffusion models for code generation are a big deal. If they are using transformers this would likely fall into the DiT bucket (diffusion transformers). I had previously worked on use cases that leveraged U-Net diffusion several years ago and there was quite a bit of inte…

I am not sure.

In principle one would imagine that models of this type would have an advantage-- you can use information from both the left and right, etc. and in practice I've found LLaDA to be impressive considering its size and my assumption that they have had small training resources, but they are behind in perplexity, and I think this is unavoidable. They also become rather fixed early, so I don't believe fully in these hopes to be able to really correct text deeply (although they will of course be able to correct their partially completed texts to some degree, especially when it's just a word or two that are wrong, but I believe that the words that are wrong basically need to get masked simultaneously, so 1/masking_probability^2, and 1/masking_probability^3 for three and so on).

Despite this I've been happy with the practical results I've seen during my experimentation.

Re: Gemini Diffusion

#117

Post by internet_points »

manmal wrote: post #54
airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

They could read the whole git history and have all issue tracker tickets in the context, and maybe even recordings from meetings. It remains to be seen though if such large context will yield usable results.

That also leads to more noise and opportunities to get lost in the woods.

Re: Gemini Diffusion

#118

Post by internet_points »

wenc wrote: post #74
efitz wrote: post #66

Earlier quoted context omitted.

No. Spell check frequently still gets things wrong if the word is spelled correctly and the sentence is grammatically correct but the wrong word was used.

Can you give me an example? Spell check only checks if a word is in dictionary. It doesn’t check grammar or context.

Finnish would like a word. Take a random noun like kauppa "shop". It has at least 6000 forms: https://flammie.github.io/omorfi/genkau3.html and that's excluding compounds (written as one word in Finnish) like "bookshop" or "shop-manager" etc. etc. And then you have loan words and slang, derivations into other words classes; all of this is impossible to compactly represent in a full-form word list.

Now consider the many other languages of that family ( https://en.wikipedia.org/wiki/Uralic_languages ) – they also have this extreme potential for inflections, but next to no online resources to train language models on or even scrape decent wordlists from.

Re: Gemini Diffusion

#119

Post by [deleted] »

airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

[deleted]

Re: Gemini Diffusion

#120

Post by Timwi »

wenc wrote: post #74
efitz wrote: post #66

Earlier quoted context omitted.

No. Spell check frequently still gets things wrong if the word is spelled correctly and the sentence is grammatically correct but the wrong word was used.

Can you give me an example? Spell check only checks if a word is in dictionary. It doesn’t check grammar or context.

That's exactly what they're saying. If you write “the work required deep incite”, a traditional spell checker won't catch the mistake (but people consider it a spelling error).
Post reply on HN