Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini Diffusion

simonwillison.net

Post reply on HN
Print view View original

241–250 of 252 posts

Re: Gemini Diffusion

#241

Post by simonw »

otabdeveloper4 wrote: post #236
johnisgood wrote: post #154

Earlier quoted context omitted.

1k LOC is perfectly fine, I did not experience issues with Claude with most (not all) projects around ~1k LOC.

Actual projects where you'd want some LLM help start with millions of lines of code, not thousands. With 1k lines of code you don't need an LLM, the entire source code can fit in one intern's head.

Have you tried Claude Code yet?

Even with it's 200,000 token limit it's still really impressive at diving through large codebases using find and grep.

Re: Gemini Diffusion

#242

Post by ckw »

westoncb wrote: post #123
airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

The trick to this is you've got to talk to them and share this information in the same way. I can give an example. These days my main workflow is as follows: if I have some big feature/refactor/whatever I'm going to work on I'll just start talking to o3 about it essentially as if it was a coworker and (somewhat painstakingly) paste in relevant source files it needs for context. We'll have a high-level discussion abou…

I do the same thing, though sometimes I take one extra step to elaborate on the first implementation plan ‘in minute detail such that a weaker model could successfully implement it’, with deep research selected.

Re: Gemini Diffusion

#243

Post by tacitusarc »

macleginn wrote: post #139
thinkingemote wrote: post #103

Earlier quoted context omitted.

"Bob went to Venice to pick up the doge." Where doge is both the name of a title (like duke) but it is misspelt "dog". The use of "Venice" where doge's are could increase a the likelihood of a smarter spell check keeping doge and not correcting to dog. Looking at a wider context might see that Bob is talking about a pupper. A simpler example would be "spell cheque"

A spelling error, using one dictionary definition, is "an error in the conventionally accepted form of spelling a word" --- mistaking one word for another does not fall under this definition. It is true that we now expect spell checkers to do grammatical checking as well, but a pure spell checker can indeed rely on a wordlist for English (this wouldn't work in languages with more developed morphology and/or frequent…

I don’t think I agree with your interpretation of the definition.

If I spell the word “pale” as “pal”, that is not an acceptable spelling for the word “pale”, even if it is coincidentally the acceptable spelling for an entirely different word.

If I asked a human editor to spellcheck the sentence: “His mouth dropped and he turned pal.”, the editor would correctly indicate I had misspelled the word.

Spellcheck hasn’t done this in the past because it can be quite difficult. But that’s a limitation of computer capability, not functionality bounded by the definition of the term “spellcheck”.

Re: Gemini Diffusion

#245

Post by scotty79 »

grumbelbart2 wrote: post #177
scotty79 wrote: post #159

Earlier quoted context omitted.

Attention is just completely arbitrary way to split the network so the learning can be parallelized. What contributed more towards success in my opinion are "shortcut connections" through layers which enable more influence on early layers during learning.

> What contributed more towards success in my opinion are "shortcut connections" through layers which enable more influence on early layers during learning. For those who don't know, that is the idea behind ResNet (He et al., Deep Residual Learning for Image Recognition, https://arxiv.org/abs/1512.03385 ), one of the most influential papers in deep learning of all time. Residual connections make it possible to train…

It's really nice to have your personal intuitions in a field you barely know confirmed by research.

Re: Gemini Diffusion

#246

Post by peter_d_sherman »

>"Traditional autoregressive language models generate text one word – or token – at a time. This sequential process can be slow, and limit the quality and coherence of the output.

Diffusion models work differently. Instead of predicting text directly, they learn to generate outputs by refining noise, step-by-step. This means they can iterate on a solution very quickly and error correct during the generation process."

It would seem that diffusion / noise filtering / processing -- would be more parallelizable than traditional autoregressive AI large language models...

That might be an interesting area of study... the parallelization of such algorithms...

Re: Gemini Diffusion

#247

Post by otabdeveloper4 »

exe34 wrote: post #240
otabdeveloper4 wrote: post #239

Earlier quoted context omitted.

Anything less is not a "project", it's a "file".

That's right, there is no true Scotsman!

Incorrect attempt as fallacy baiting.

If your repo map fits into 1000 tokens then your repo is small enough that you can just concatenate all the files together and feed the result as one prompt to the LLM.

No, current LLM technology does not allow to process actual (i.e. large) repos.

Re: Gemini Diffusion

#248

Post by simonw »

otabdeveloper4 wrote: post #247
exe34 wrote: post #240

Earlier quoted context omitted.

That's right, there is no true Scotsman!

Incorrect attempt as fallacy baiting. If your repo map fits into 1000 tokens then your repo is small enough that you can just concatenate all the files together and feed the result as one prompt to the LLM. No, current LLM technology does not allow to process actual (i.e. large) repos.

Where's your cutoff for "large"?

Re: Gemini Diffusion

#249

Post by landl0rd »

westoncb wrote: post #123
airstrike wrote: post #3

That's...ridiculously fast. I still feel like the best uses of models we've seen to date is for brand new code and quick prototyping. I'm less convinced of the strength of their capabilities for improving on large preexisting content over which someone has repeatedly iterated. Part of that is because, by definition, models cannot know what is not in a codebase and there is meaningful signal in that negative space. En…

The trick to this is you've got to talk to them and share this information in the same way. I can give an example. These days my main workflow is as follows: if I have some big feature/refactor/whatever I'm going to work on I'll just start talking to o3 about it essentially as if it was a coworker and (somewhat painstakingly) paste in relevant source files it needs for context. We'll have a high-level discussion abou…

This is absolutely the best way to do it. However it's also infeasible for number-of-queries-based quota like most front-ends have. And of course running through API for models like o3 and 4-opus is basically always way more expensive. Hence the desire for one-shotting stuff.

Re: Gemini Diffusion

#250

Post by landl0rd »

Incipient wrote: post #65
8n4vidtmkvmk wrote: post #46

Earlier quoted context omitted.

That's not been my experience so far. LLMs are good at mimicking existing good, it doesn't usually bring in new things when not asked. Sometimes I have to go out of my way to point to other bits of code in the project to copy from because it hasn't ingested enough of the codebase. That said, a negative prompt like we have in stable diffusion would still be very cool.

I'm in the camp of 'no good for existing'. I try to get ~1000 line files refactored to use different libraries, design paradigms, etc and it usually outputs garbage - pulling db logic into the UI, grabbing unrelated api/function calls, to entirely just corrupting the output. I'm sure there is a way to correctly use this tool, so I'm feeling like I'm "just holding it wrong".

I'll back this up. I feel constantly gaslit by people who claim they get good output.

I was hacking on a new project and wanted to see if LLMs could write some of it. So I picked an LLM friendly language (python). I picked an LLM friendly DB setup (sqlalchemy and postgres). I used typing everywhere. I pre-made the DB tables and pydantic schema. I used an LLM-friendly framework (fastapi). I wrote a few example repositories and routes.

I then told it to implement a really simple repository and routes (users stuff) from a design doc that gave strict requirements. I got back a steaming pile of shit. It was utterly broken. It ignored my requirements. It fucked with my DB tables. It fucked with (and broke) my pydantic. It mixed db access into routes which is against the repository pattern. Etc.

I tried several of the best models from claude, oai, xai, and google. I tried giving it different prompts. I tried pruning unnecessary context. I tried their web interfaces and I tried cursor and windsurf and cline and aider. This was a pretty basic task I expect an intern could handle. It couldn't.

Every LLM enthusiast I've since talked to just gives me the run-around on tooling and prompting and whatever. "Well maybe if you used this eighteenth IDE/extension." "Well maybe if you used this other prompt hack." "Well maybe if you'd used a different design pattern."

The fuck?? Can vendors not produce a coherent set of usage guidelines? If this is so why isn't there a set of known best practices? Why can't I ever replicate this? Why don't people publish public logs of their interactions to prove it can do this beyond a "make a bouncing ball web game" or basic to-do list app?

Post reply on HN