Live data from Hacker News

A study on robustness and reliability of large language model code generation

arxiv.org

181–190 of 229 posts

Re: A study on robustness and reliability of large language model code generation

#181
post #90

Earlier quoted context omitted.

The over-hype is causing it to look under-hyped. 48% of the code being freaking correct from 0% about 5 years ago is huge. With everyone and all the news talking about it constantly inevitably people are going to start rolling their eyes. This is bias. 48% is half way to 100%. Once it reaches 100% you don't have a job. You realize that right? It's halfway their to taking your job and you're underwhelmed. The bias is…

Even if it were possible to get to 100% (it isn't), I'd still have a job. It's a great assistant. I use it. It's not taking anybody's job.

It's taking your assistants job.

Re: A study on robustness and reliability of large language model code generation

#182

Run the code it writes, if it gives an error, paste it into the chat and 90% of the time the LLM can fix the issue. People are missing the point here - it's not about writing code in one-shot. An LLM-enabled loop can generate the code and test and refine it until it works

Someone posted a transcript of having chatgpt write them a bash script with the comment "see how much easier this was than figuring it out" and it took like a dozen tries of pasting error messages back in. I was infuriated just reading it. I cannot imagine trying to develop this way. But if people like it, whatever blows your hair back. Allegedly the benefit is it will do the boring stuff like write the error handlin…

I think you’re missing the point here.

Language models scale.

It doesn’t matter if a single pass doesn’t solve the problem, has syntax errors, etc. A single pass costs a fraction of a cent.

You can just automate the process of code -> create variant -> fix from LLM -> apply deterministic tests until the code at least compiles -> pass it to the user; the fact you can’t do that with chatgpt is just because it’s not a coding tool.

Heck, there are already companies doing exactly this with vulnerability scanners (ie. the deterministic feedback loop) to suggest security fixes for code.

You just repeat a heap of times until you get a solution that passes all the scanners.

If it takes 10 tries, or 100 tries, it still is zero effort from a human.

Of course, whether the result does the right thing is another matter, but the frustrating ergonomics of copy-paste cycle is because of the ui, not the technology.

You will see “surprisingly good” output from professional tools in this space (we already are); but a lot of it is not magic sauce; it’s just the same tools, run multiple times, in a way that saves you doing it manually and just shows the best results, with a pretty ribbon on it.

Re: A study on robustness and reliability of large language model code generation

#183
post #174

Earlier quoted context omitted.

I see humans do this all the time, especially abuse of exception handling, even among “Senior” developers. They don’t have a semantic understanding of what they are doing or why they are creating a race conditions or creating perverse control flow logic N layers down in the stack. The fact that researchers get it wrong is, well, unsurprising. LLMs might actually be an improvement.

Hi, do you have a recommended read for those of us who might inadvertently create race conditions?

Now hopefully I won't get horribly dinged for mistakes and poor advice here. What I am trying to say is that I too read "even senior devs don't understand the race conditions they create downstream." And I thought - oh God, don't I.

But five minutes thought can help you walk through most issues. For most applications most of the time you can reason your way through without fear, and when you do encounter gnarly problems they often can go away by redesigning your application! Often the problems you encounter you caused. retrace your steps and find an easier path. Save the hard thinking for genuine problems / value creation.

So race conditions are simply when two processes / threads are likely to affect a single resource. In this case it's a file - and the problem is test if a file exists, then if it does not, create it and then write to it.

If two threads do this, say a log file, the first one creates the file and logs it's important stuff, the second then creates it again wiping out the first log data.

Solutions in this area include

- create as append file (the concept is basically deviates old because this is a decades old problem)

- avoid sharing resources. for logging log to per thread locations. Not always possible but you sure as hell can minimise this to one or two resources you must share.

- hand off creating files to a seperate part of the application. There is a balance between "scripting" and "application" and using small little library functions to do in one line indirectly something that also takes one line using the methods shown in the docs.

- handing off batons / mutexes etc etc. This gets wildly complex. Honestly given a world of async libraries, Erlang, and distributed computing, if you find yourself having to use multi-threading think very carefully if this is the right approach.

Re: A study on robustness and reliability of large language model code generation

#184
> we propose a dataset RobustAPI for evaluating the reliability and robustness of code generated by LLMs

While such a set may be interesting to study in itself, it does not seem like a reasonable dataset for evaluating LLM-generated code overall. Of course, if significant issues are found with this set (and they are) - then there are significant issues overall (and there are).

Re: A study on robustness and reliability of large language model code generation

#185

I feel like AI for programming has been so overhyped. I've attempted to use ChatGPT for programming so many times, and almost every time it's just wasted my time. Giving me outright lies or generating stuff that looks right but doesn't work. And just throwing out it's answer and starting from scratch is faster than fixing its output. I've only found it useful for explaining basic terminology or concepts for a topic I…

The over-hype is causing it to look under-hyped. 48% of the code being freaking correct from 0% about 5 years ago is huge. With everyone and all the news talking about it constantly inevitably people are going to start rolling their eyes. This is bias. 48% is half way to 100%. Once it reaches 100% you don't have a job. You realize that right? It's halfway their to taking your job and you're underwhelmed. The bias is…

I think you're misunderstanding the problem here. It could be 97% correct and it would still be unusable. Getting things to 100% is so hard it might as well be impossible, and the fact that you always need a person to check all the output limits the utility.

Re: A study on robustness and reliability of large language model code generation

#186
post #102

Earlier quoted context omitted.

You assume that getting from “48% as good” as a human programmer (on an undoubtedly biased sample) to “100% as good” is about equally hard as getting from 0% to 48%. That seems a pretty big assumption. Many programmers also do work that doesn’t just involve stitching APIs together; GPT consistently fails badly when forced to reason, making it useless for many tasks. For example, ask it to implement a common algorithm…

>You assume that getting from “48% as good” as a human programmer (on an undoubtedly biased sample) to “100% as good” is about equally hard as getting from 0% to 48%. That seems a pretty big assumption. I meant 38% aka 40% made a mistake on my math. It's not a big assumption. It's the most reasonable assumption. When you drive 50% of 10 miles the next 50% takes the same amount of time. It's the default assumption. I…

> It's not a big assumption. It's the most reasonable assumption. When you drive 50% of 10 miles the next 50% takes the same amount of time. It's the default assumption.

And it's wrong, as anyone with experience in ML will tell you. 60% is easy. 70-85% is not too bad. 95% is hard. 100% is effectively impossible. This is the core problem of ML systems. They're good enough for 90% of cases but that last 10% can be incredibly important, to the point that the systems become unusable.

Re: A study on robustness and reliability of large language model code generation

#187

Earlier quoted context omitted.

First off, you did not address most of my point. To answer your complaint, I am absolutely looking at reality. Please point us to an actual real-world professional programming job, matching the criteria I list above, for which a current LLM can economically replace a human for over a 5-year period. My believe is that there are exactly zero such jobs. If your claim is that we're at least at 1%, then you're claiming th…

>My believe is that there are exactly zero such jobs. If your claim is that we're at least at 1%, then you're claiming that there are at least 269k fully automatable programmer jobs [1]. It shouldn't be hard for you to find at least one to start. If I built an entire car but I am missing the key. I cannot drive the car but the car is 99% complete. It's just missing the key. So what happens is, it can replace 0% of hu…

[flagged]

Re: A study on robustness and reliability of large language model code generation

#188

Run the code it writes, if it gives an error, paste it into the chat and 90% of the time the LLM can fix the issue. People are missing the point here - it's not about writing code in one-shot. An LLM-enabled loop can generate the code and test and refine it until it works

Someone posted a transcript of having chatgpt write them a bash script with the comment "see how much easier this was than figuring it out" and it took like a dozen tries of pasting error messages back in. I was infuriated just reading it. I cannot imagine trying to develop this way. But if people like it, whatever blows your hair back. Allegedly the benefit is it will do the boring stuff like write the error handlin…

Man I sure love running buggy bash scripts on my system just to see what happens.

Re: A study on robustness and reliability of large language model code generation

#189

Earlier quoted context omitted.

The over-hype is causing it to look under-hyped. 48% of the code being freaking correct from 0% about 5 years ago is huge. With everyone and all the news talking about it constantly inevitably people are going to start rolling their eyes. This is bias. 48% is half way to 100%. Once it reaches 100% you don't have a job. You realize that right? It's halfway their to taking your job and you're underwhelmed. The bias is…

> 48% if half way to 100% … It's halfway their to taking your job and your underwhelmed. You’re implying 100% reliability is actually attainable. If that were the case, wouldn’t that mean the halting problem would have been solved by AI? I’m not an expert but I’ve heard that’s like one of those fundamental laws of information theory that really can’t be broken. > It's typical. It's like an indie band is only popular…

I agree with the majority of your well considered comment. I'm okay with LLMs being really unreliable though. I think if they actually worked, it would be an unmitigated disaster for workers globally.

Re: A study on robustness and reliability of large language model code generation

#190

Earlier quoted context omitted.

Someone posted a transcript of having chatgpt write them a bash script with the comment "see how much easier this was than figuring it out" and it took like a dozen tries of pasting error messages back in. I was infuriated just reading it. I cannot imagine trying to develop this way. But if people like it, whatever blows your hair back. Allegedly the benefit is it will do the boring stuff like write the error handlin…

I think you’re missing the point here. Language models scale. It doesn’t matter if a single pass doesn’t solve the problem, has syntax errors, etc. A single pass costs a fraction of a cent. You can just automate the process of code -> create variant -> fix from LLM -> apply deterministic tests until the code at least compiles -> pass it to the user; the fact you can’t do that with chatgpt is just because it’s not a c…

Sounds kind of silly, just run the thing till it gives the right answer ? How do you know each pass has not introduced a new problem ?
Post reply on HN