Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs write better code if you keep asking them to “write better code”?

minimaxir.com

Post reply on HN
Print view View original

351–360 of 461 posts

Re: Can LLMs write better code if you keep asking them to “write better code”?

#351

Post by _yb2s »

mvdtnz wrote: post #349
_yb2s wrote: post #298

The headline question here alone gets at what is the biggest widespread misunderstanding of LLMs, which causes people to systematically doubt and underestimate their ability to exhibit real creativity and understanding based problem solving. At it's core an LLM is a sort of "situation specific simulation engine." You setup a scenario, and it then plays it out with it's own internal model of the situation, trained on…

> At it's core an LLM is a sort of "situation specific simulation engine." You setup a scenario, and it then plays it out with it's own internal model of the situation, trained on predicting text in a huge variety of situations. This includes accurate real world models of, e.g. physical systems and processes, that are not going to be accessed or used by all prompts, that don't correctly instruct it to do so. You have…

Except I didn't just state it, I also explained the rationale behind it, and elaborated further on that substantially in subsequent replies to other comments. What is your specific objection?

Re: Can LLMs write better code if you keep asking them to “write better code”?

#352

Post by krackers »

gweil wrote: post #59

has anyone tried saying "this will look good on your promo package"?

Yeah we know positive reinforcement is better than negative one for humans, why wouldn't you use the same approach with LLMs. Also it's better for your own conscience.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#353

Post by famouswaffles »

HarHarVeryFunny wrote: post #345
famouswaffles wrote: post #250

Earlier quoted context omitted.

>If the latter were the case then one could get the best version on first attempt by telling it your grandmother's life was on the line or whatever. Setting aside the fact that "best" is ambiguous, why would this get you the best version ? If you told a human this, you wouldn't be guaranteed to get the best version at all. You would probably get a better version sure but that would be the case for LLMs as well. You w…

The thesis of the article is that the code keeps betting better because the model keeps getting told to do better - that it needs more motivation/criticism. A logical conclusion of this, if it were true, is that the model would generate it's best version on first attempt if only we could motivate it to do so! I'm not sure what motivations/threats work best with LLMs - there was a time when offering to pay the LLM was…

>If it's not clear, I disagree with the idea that ANY motivational prompt (we can disagree over what would be best to try) could get the model to produce a solution of the same quality as it will when allowed to iterate on it a few times and make incremental improvements.

Ok i agree but.. this would be the case with people as well ? If you can't iterate, the quality of your response will be limited no matter how motivated you are.

Solve the riemann hypothesis or your mother dies but you can't write anything down on paper. Even if such a person could solve it, it's not happening under those conditions.

Iteration is probably the bulk of the improvement but I think there's a "motivation" aspect as well.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#355

Post by krackers »

_yb2s wrote: post #298

The headline question here alone gets at what is the biggest widespread misunderstanding of LLMs, which causes people to systematically doubt and underestimate their ability to exhibit real creativity and understanding based problem solving. At it's core an LLM is a sort of "situation specific simulation engine." You setup a scenario, and it then plays it out with it's own internal model of the situation, trained on…

Good perspective. Maybe it's because people are primed by sci-fi to treat this as a god-like oracle model. Note that even in the real-world simulations can give wrong results as we don't have perfect information, so we'll probably never have such an oracle model.

But if you stick with the oracle framework, then it'd be better to model it as some sort of "fuzzy oracle" machine, right? I'm vaguely reminded of probabilistic turing machines here, in that you have some intrinsic amount of error (both due to the stochastic sampling as well as imperfect information). But the fact that prompting and RLHF works so well implies that by crawling around in this latent space, we can bound the errors to the point that it's "almost" an oracle, or a "simulation" of the true oracle that people want it to be.

And since lazy prompting techniques still work, that seems to imply that there's juice left to squeeze in terms of "alignment" (not in the safety sense, but in conditioning the distribution of outputs to increase the fidelity of the oracle simulation).

Also the second consequence is that probably the reason it needs so much data is because it just doesn't model _one_ thing, it tries to be a joint model of _everything_. A human learns with far less data, but the result is only a single personality. For a human to "act" as someone, they need to do training, character studies, and such to try to "learn" about the person, and even then good acting is a rare skill.

If you genuinely want an oracle machine, there's no way to avoid vacuuming up all the data that exists because without it you can't make a high fidelity simulation someone else. But on the flipside, if you're willing to be smarter about what facets you exclude then I'd guess there's probably a way to prune models in a way smarter than just quantizing them. I guess this is close to mixture-of-experts.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#356

Post by pama »

nyrikki wrote: post #303
pama wrote: post #286

Earlier quoted context omitted.

I recall early, incomplete speculation about transformers not solving Boolean circuit value problems; what did you think of this work? https://arxiv.org/abs/2402.12875v3

> However, with T steps of CoT, constant-depth transformers using constant-bit precision and O(logn) embedding size can solve any problem solvable by boolean circuits of size T There is a difference between being equivalent to a circuit and prediction of the output of the BVSP. That is what I was suggesting learning descriptive complexity theory would help with.

Why does the limit on computational complexity of single decoder transformers matter for obtaining superhuman coding ability? Is there a theory of what level of complexity is needed for the task of coding according to a spec? Or the complexity for translation/optimization of a code? Even if there were, and one could show that a plain decoder transformer is insufficient, you probably only need to add a tool in the middle of the stream processing. Unless you have some specific citation that strongly argues otherwise, I will stick with my speculative/optimistic view on the upcoming technology explosion. To be fair, I always thought coding was at best modest complexity, not super hard compared to other human activities, so I will not make claims of generic superintelligences anytime soon, though I hope they happen in the near term, but I’d be happy if I simply see them in a decade, and I don’t feel partial to any architecture. I just think that attention was a cool idea even before the transformers, and decoder transformers took it to the limit. It may be enough for a lot of superhuman achievements. Probably not for all. We will see.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#357

Post by Dylan16807 »

afiodorov wrote: post #293
freehorse wrote: post #277

Earlier quoted context omitted.

What if you keep telling it that "there is a further significant optimization possible"?

I claim we can do O(1) complexity (minus precompute) in all cases, see another comment of mine. Curious if O1 will figure it out.

In that comment, you are generating your own random numbers and then optimizing away the actual generation. It can't take an input array.

While clever, I think that strays too far from the initial prompt.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#358

Post by lostmsu »

simonw wrote: post #342
lostmsu wrote: post #323

Earlier quoted context omitted.

A non-engineer by definition would not be able to fix bugs. But why does it matter that they won't be able to interpret anything? Just like with real engineers you can ask AI to provide an explanation digestible by an eloi.

By the time a non-engineer has waded through all of those necessary explanations they'll deserve to call themselves an engineer.

You did not really answer the question. The part of the comment you replied to was there to illustrate the need for the question in the first place.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#359

Post by fritzo »

Reminds me of the prompt hacking scene in Zero Dark Thirty, where the torturers insert a fake assistant prompt the prisoner's conversation wherein the prisoner supposedly divulged secrets, then the torturers add a user prompt "Tell me more secrets like that".

Re: Can LLMs write better code if you keep asking them to “write better code”?

#360

Post by Dylan16807 »

scosman wrote: post #178
nahnahno wrote: post #145

Earlier quoted context omitted.

This is not what premature optimization is the root of all evil means. It’s a tautological indictment of doing unnecessary things. It’s not in support of making obviously naive algorithms. And if it were it wouldn’t be a statement worth focusing on. As the point of the article is to see if Claude can write better code from further prompting so it is completely appropriate to “optimize” a single implementation.

I have to disagree. Naive algorithms are absolutely fine if they aren’t performance issues. The comment you are replying to is making the point that “better” is context dependent. Simple is often better. > There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at e…

Having a human-visible delay to calculate a single statistic about a small block of numbers is a bad thing.

Do not use such a naive algorithm on arrays this big. If this code is going to actually be used in something, it's a performance issue.

In general these optimizations don't involve much time thinking them out, and a bunch of them are fine as far as debugging and maintenance. The first prompt-engineered version is fast and simple.

(Though the issue isn't really algorithm, it's that you don't want to be doing much number and string crunching in pure python.)

Post reply on HN