Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs write better code if you keep asking them to “write better code”?

minimaxir.com

Post reply on HN
Print view View original

381–390 of 461 posts

Re: Can LLMs write better code if you keep asking them to “write better code”?

#381

Post by [deleted] »

ThrowawayR2 wrote: post #212
btbuildem wrote: post #140

I've noticed this with GPT as well -- the first result I get is usually mediocre and incomplete, often incorrect if I'm working on something a little more obscure (eg, OpenSCAD code). I've taken to asking it to "skip the mediocre nonsense and return the good solution on the first try". The next part is a little strange - it arose out of frustration, but it also seems to improve results. Let's call it "negative incent…

We've entered the voodoo witch doctor phase of LLM usage: " Enter thee this arcane incantation along with thy question into the idol and, lo, the ineffable machine spirits wilt be appeased and deign to grant thee the information thou hast asked for. "

[deleted]

Re: Can LLMs write better code if you keep asking them to “write better code”?

#382

Post by petee »

btbuildem wrote: post #140

I've noticed this with GPT as well -- the first result I get is usually mediocre and incomplete, often incorrect if I'm working on something a little more obscure (eg, OpenSCAD code). I've taken to asking it to "skip the mediocre nonsense and return the good solution on the first try". The next part is a little strange - it arose out of frustration, but it also seems to improve results. Let's call it "negative incent…

I've found just being friendly, but highly critical and suspicious, gets good results.

If you can get it to be wordy about "why" a specific part of the answer was given, it often reveals what its stumbling on, then modify your prompt accordingly.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#383

Post by maxlamb »

btbuildem wrote: post #140

I've noticed this with GPT as well -- the first result I get is usually mediocre and incomplete, often incorrect if I'm working on something a little more obscure (eg, OpenSCAD code). I've taken to asking it to "skip the mediocre nonsense and return the good solution on the first try". The next part is a little strange - it arose out of frustration, but it also seems to improve results. Let's call it "negative incent…

So you’re telling me this clip isn’t even satire: https://youtube.com/shorts/64TNGvCoegE

Re: Can LLMs write better code if you keep asking them to “write better code”?

#384

Post by weaksauce »

blueaquilae wrote: post #377
cultureulterior wrote: post #265

Earlier quoted context omitted.

O1 found it. https://chatgpt.com/share/67782b6b-6248-8012-882d-238b600ef9...

Did it found it before the HN comment? O1 has access to the web so I'm just asking

you could maybe test that with many comments describing a false approach.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#385

Post by emseetech »

evilduck wrote: post #374
emseetech wrote: post #244

Earlier quoted context omitted.

I give every AI tool a college try and have since the copilot beta. I’m simply not interested in having these tools type for me. Typing is nowhere near the hardest part of my job and I find it invaluable as a meditative state for building muscle memory for the context of what I’m building. Taking shortcuts has a cost I’m not willing to pay.

If your position is to find fault then that’s what you will accomplish.

I'm speaking from experience and observation of the past two years of LLM assistants of various kinds that outsourcing code production will atrophy your skills generally and will threaten your contextual understanding of a codebase specifically over the long term.

If that's a risk you're willing to take for the sake of productivity, that can be a reasonable tradeoff depending on your project and career goals.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#386

Post by GuB-42 »

gjm11 wrote: post #137
dgacmu wrote: post #80

I'm amused that neither the LLM or the author identified one of the simplest and most effective optimizations for this code: Test if the number is max _before_ doing the digit sum. It's a free 5.5x speedup that renders some of the other optimizations, like trying to memoize digit sums, unnecessary. On an m1 macbook pro, using numpy to generate the random numbers, using mod/div to do digit sum: Base: 55ms Test before…

There's another, arguably even simpler, optimization that makes me smile. (Because it's silly and arises only from the oddity of the task, and because it's such a huge performance gain.) You're picking 1,000,000 random numbers from 1 to 100,000. That means that any given number is much more likely to appear than not. In particular, it is very likely that the list contains both 3999 (which is the smallest number with…

In fact, we could simply check for the 3 smallest and the 3 highest numbers and ignore the rest.

Assuming the numbers are really random, that's a probability of 10^-13. That probability is at the point where we are starting to think about errors caused by cosmic rays. With a bit more numbers, you can get to the point where the only way it can fail is if there is a problem with the random number generation or an external factor.

If it was something like a programming contest, I would just do "return 95931" and hope for the best. But of course, programming contests usually don't just rely on random numbers and test edge cases.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#388

Post by weakfish »

_yb2s wrote: post #350
CharlesW wrote: post #348

Earlier quoted context omitted.

> I'm not sure if you read the entirety of my comment? I did, and I tried my best to avoid imposing preconceived notions while reading. You seem to be equating "being able to predict the next symbol in a sequence" with "possessing a deep causal understanding of the real-world processes that generated that sequence", and if that's an inaccurate way to characterize your beliefs I welcome that feedback. Before you judge…

> You seem to be equating "being able to predict the next symbol in a sequence" with "possessing a deep causal understanding of the real-world processes that generated that sequence" More or less, but to be more specific I would say that increasingly accurately predicting the next symbols in a massive set of diverse sequences, which explain a huge diversity of real world events described in sequential order, requires…

> specific I would say that increasingly accurately predicting the next symbols in a massive set of diverse sequences, which explain a huge diversity of real world events described in sequential order, requires increasingly accurate models of the underlying processes of said events

But it doesn’t - it’s a statistical model using training data, not a physical or physics model, which you seem to be equating it to (correct me if I am misunderstanding)

And in response to the other portion you present, an LLM fundamentally can’t be alien because it’s trained on human produced output. In a way, it’s a model of the worst parts of human output - garbage in, garbage out, as they say - since it’s trained on the corpus of the internet.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#389

Post by weakfish »

nextaccountic wrote: post #156
b112 wrote: post #82

Earlier quoted context omitted.

I wish people would understand what a large language model is. There is no thinking. No comprehension. No decisions. Instead, think of your queries as super human friendly SQL. The database? Massive amounts of data boiled down to unique entries with probabilities. This is a simplistic, but accurate way to think of LLMs. So how much code is on the web for a particular problem solve? 10k blog entries, stackoverflow res…

> I wish people would understand what a large language model is. There is no thinking. No comprehension. No decisions. > Instead, think of your queries as super human friendly SQL. Ehh this might be true in some abstract mathy sense (like I don't know, you are searching in latent space or something), but it's not the best analogy in practice. LLMs process language and simulate logical reasoning (albeit imperfectly).…

I don’t understand the axiom that language skills give reasoning for free, can you expand? That seems like a logical leap to me

Re: Can LLMs write better code if you keep asking them to “write better code”?

#390

Post by econ »

mulmen wrote: post #318
bennythomsson wrote: post #287

Earlier quoted context omitted.

Amazing. Next step would be to propose hardcoding 99930-3999 as the O(1) result and live with the output just being wrong sometimes. The bug rate is then in the ballpark of most modern software, including LLMs', so I'd say ship it.

Doesn’t this line of thinking constantly redefine success until all software is only bugs?

Keep the secret!
Post reply on HN