Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs write better code if you keep asking them to “write better code”?

minimaxir.com

Post reply on HN
Print view View original

401–410 of 461 posts

Re: Can LLMs write better code if you keep asking them to “write better code”?

#401

Post by animal531 »

I've been working on some low level Unity C# game code and have been using GPT to quickly implement certain algorithms etc.

One time it provided me with a great example, but then a few days later I couldn't find that conversation again in the history. So I asked it about the same question (or so I thought) and it provided a very subpar answer. It took me at least 3 questions to get back to that first answer.

Now if it had never provided me with the first good one I'd have never known about the parts it skipped in the second conversation.

Of course that could happen just as easily by having used google and a specific reference to write your code, but the point I'm trying to make is that GPT isn't a single entity that's always going to provide the same output, it can be extremely variable from terrible to amazing at the end of the day.

Having used google for many years as a developer I'm much better at asking it questions than say people in the business world is, I've seen them struggling to question it and far too easily giving up. So I'm quite scared to see what's going to happen once they really start to use and rely on GPT, the results are going to be all over the place.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#402

Post by fragmede »

hollywood_court wrote: post #253
ThrowawayTestr wrote: post #251

Earlier quoted context omitted.

This makes me sad. Have you tried being really nice and supportive instead? I really don't want to have to yell at my computer for it to work :(

Yes and it didn't work. I've actually got Cursor/Claude to curse back at me. Well, not AT me, but it used profanity in it's response once it realized that it was going around in circles and recreating the same errors.

Shit, that makes me a lot more worried for my job than any programming test. My ability to swear at the computer, and not user the word delve, is what sets l us apart from AI. if they can do that, what hope is there for the future?

Re: Can LLMs write better code if you keep asking them to “write better code”?

#403

Post by fragmede »

NitpickLawyer wrote: post #76
codesections wrote: post #20

> “Planning” is a long-used trick to help align LLM output for a first pass — the modern implementation of “let’s think step by step.” I hadn't seen this before. Why is asking for planning better than asking it to think step by step?

This is how aider becomes really good: - start by "chatting" with the model and asking for "how you'd implement x y z feature, without code". - what's a good architecture for x y z - what are some good patterns for this - what are some things to consider when dealing with x y z - what are the best practices ... (etc) - correct / edit out some of the responses - say "ok, now implement that" It's basically adding stuff…

aider has an /architect command to help with that type of thinking.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#404

Post by fragmede »

yubrshen wrote: post #347

When asking LLM repeated improving or adding a new feature in a codebase, the most frustration risk is that LLM might wipe out already working code! What are your strategies to prevent such destructions of LLM?

same thing a human does, stick it in git. tools like aider use git, along with heuristics on LLM output. If the working code is wiped out, give it a few more prompts to let it fix it, or revert ban to a known good/working copy.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#405

Post by brabel »

ActivePattern wrote: post #331
brabel wrote: post #270

Earlier quoted context omitted.

Did you measure it? I would expect using % would ruin your performance as it's slow, even if it allows you to avoid doing a bunch of sums (which are fast).

Doing a single modulo 9 operation is much faster than summing a d-digit number, which requires d modulo 10s, d divide 10s, and d sums.

You can do the sum by looking directly at the digits in the string, no need for module at all.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#406

Post by withinboredom »

IanCal wrote: post #361
withinboredom wrote: post #174

Earlier quoted context omitted.

The problem is that you can do this for specific functions/methods, but you cannot do this for a PROGRAM. All programs are "maybe", by definition. You want it to run until you tell it to stop, but you may never tell it to stop. Ergo, all programs have some sort of infinite loop in them somewhere, even if it is buried in your framework or language runtime.

I really don't think that's the definition of a program. > You want it to run until you tell it to stop, No? Many programs I don't want to run until I tell them to stop. Even then, this reduces it to irrelevance.

Yeah, sorry, I wasn’t clear: not the user, the programmer. This is true for almost all programs. Even a simple “print hello world” involves at least one intentional infinite loop: sending bytes to the buffer. The buffer could remain full forever.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#407

Post by Yoric »

withinboredom wrote: post #173
Yoric wrote: post #53

Earlier quoted context omitted.

> It seems like with Rust it's not quite aware of which patterns to use, especially when the actual changes required may span multiple files due to the way memory management is structured. What do you mean? Memory management is not related to files in Rust (or most languages).

When did they say that?

As far as I can tell, in the extract I quoted. Do you interpret it differently?

Re: Can LLMs write better code if you keep asking them to “write better code”?

#408

Post by withinboredom »

Yoric wrote: post #407
withinboredom wrote: post #173

Earlier quoted context omitted.

When did they say that?

As far as I can tell, in the extract I quoted. Do you interpret it differently?

> It seems like with Rust it [the AI] is not quite aware of which patterns to use, especially when the actual changes required may span multiple files due to the way memory management is structured.

In rust, when you refactor something that deals with the borrow checker's shenanigans, you will likely have to change a bunch of files (from experience). This means that an AI will likely also have to change a bunch of files which they say the AI isn't so good at. They don't say this HAS to happen, just that it usually does because the borrow checker is an asshole.

This aligns with my experience as well, though I dealt with Rust before there was AI, so I can say little in regards to how the AI deals with that.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#409

Post by evrimoztamur »

withinboredom wrote: post #408
Yoric wrote: post #407

Earlier quoted context omitted.

As far as I can tell, in the extract I quoted. Do you interpret it differently?

> It seems like with Rust it [the AI] is not quite aware of which patterns to use, especially when the actual changes required may span multiple files due to the way memory management is structured. In rust, when you refactor something that deals with the borrow checker's shenanigans, you will likely have to change a bunch of files (from experience). This means that an AI will likely also have to change a bunch of fi…

Thank you for the breakdown, this is exactly what I meant.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#410

Post by freehorse »

Dylan16807 wrote: post #373
afiodorov wrote: post #362

Earlier quoted context omitted.

All I need is the proportion of the qualifying numbers to the input array to run the algorithm and the number of samples. Then we can sample min, max index of the qualifying array and return their difference without having to sample many times if we can derive the joined min max distribution conditional on the Bernoulli. In other words the procedure can take any input array and qualifying criteria. The joint distribu…

Sampling doesn't give you the actual answer for an actual array. If the program uses the array for multiple things, such as organizing the numbers after allocating the correct number of buckets, your method will cause logic errors and crashes. The O(1) method based on statistics only works when the function making this calculation can hide the array (or lack of array) behind a curtain the entire time. If it has to ta…

It depends on how you read the problem still. In a lot of the llms solutions the array is not provided in the solving functions but rather constructed inside (as instead of defining the function with an input and then creating a main function that would be called with no argument, construct an array and call the solving function with that as argument, as typical in python), so I assume the llm did not read it like this or also failed this aspect of the code (which was never really mentioned). It is not clear if we are given a specific array of integers or one input is an array of random variables that we need to instantiate ourselves.
Post reply on HN