Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study finds AI assistants help developers produce code likely to be buggy

theregister.com

Post reply on HN
Print view View original

141–150 of 153 posts

Re: Study finds AI assistants help developers produce code likely to be buggy

#141

Post by jrvarela56 »

dmix wrote: post #44
jrvarela56 wrote: post #6

The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just fine, my workflow went something like this: - Identify a component in a system you want to build - Give ChatGPT a list of bullets and ask it to write tests - Review tests - Run tests, give ChatGPT output of failing tests - Add the code to your codebase, run th…

I wonder if you could build a dataset of UI components (React/HTML/CSS) with a simple description and taxonomy then feed it into an AI algo

You can but this is a trickier version of the 'package context' process (see my reply to a comment above).

Speaking to others who are playing with this stuff, it seems people who use opinionated libraries (Antd, Mantine) get better results out of the box.

Re: Study finds AI assistants help developers produce code likely to be buggy

#142

Post by jrvarela56 »

jlarocco wrote: post #90
jrvarela56 wrote: post #6

The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just fine, my workflow went something like this: - Identify a component in a system you want to build - Give ChatGPT a list of bullets and ask it to write tests - Review tests - Run tests, give ChatGPT output of failing tests - Add the code to your codebase, run th…

I'm guessing it depends on the situation, but at what point does all of that become more work than just writing the tests and code yourself? Is it worth the "investment" to learn how to teach ChatGPT coding? Or will ChatGPT2 come out in six months, based on a different algorithm, and you'll back to square one teaching it over again? Personally, I find writing new code easier than editting existing code, regardless of…

It's obviously not paying off right now, but I'm assuming these models will get there in 2023 so I'm getting ready for that :P

To take into account future changes, my approach is to develop interactive models for the components that make up this system using Jupyter notebooks and https://natto.dev/. The idea is that if some part changes you can sort of test it in isolation and compare results.

Re: Study finds AI assistants help developers produce code likely to be buggy

#143

Post by mountainriver »

Salgat wrote: post #121
visarga wrote: post #104

Earlier quoted context omitted.

Expand your view. AI can write tests, read error messages, find bugs in your code, we just need to give this task. Let's think about tests. You write a function, the AI writes a few tests for you. maybe you need to add a few more. But it's better to have tests, and you might have missed one of them. Error messages - we rely on error messages to make this leap from "code parrots" to "bug free". Most of our codes fail…

All those things are susceptible to the same issue: the ML model can generate test cases and error messages that are convincing to you and me regardless of whether they're actually right. Don't get me wrong, ML will reach a point someday where it catches up to humans in this respect; this is merely a shortcoming of today's ML, not tomorrow's, and it highlights where ML's weaknesses currently lie (in contrast to more…

Yup the next iteration of ML needs to focus more on proofs. Okay you know this but how do you know it, show me the proof

Re: Study finds AI assistants help developers produce code likely to be buggy

#144

Post by Salgat »

mountainriver wrote: post #143
Salgat wrote: post #121

Earlier quoted context omitted.

All those things are susceptible to the same issue: the ML model can generate test cases and error messages that are convincing to you and me regardless of whether they're actually right. Don't get me wrong, ML will reach a point someday where it catches up to humans in this respect; this is merely a shortcoming of today's ML, not tomorrow's, and it highlights where ML's weaknesses currently lie (in contrast to more…

Yup the next iteration of ML needs to focus more on proofs. Okay you know this but how do you know it, show me the proof

It's about refining the approximations it makes until those approximations are too tiny/negligible to create failures in code.

Re: Study finds AI assistants help developers produce code likely to be buggy

#145

Post by visarga »

Salgat wrote: post #121
visarga wrote: post #104

Earlier quoted context omitted.

Expand your view. AI can write tests, read error messages, find bugs in your code, we just need to give this task. Let's think about tests. You write a function, the AI writes a few tests for you. maybe you need to add a few more. But it's better to have tests, and you might have missed one of them. Error messages - we rely on error messages to make this leap from "code parrots" to "bug free". Most of our codes fail…

All those things are susceptible to the same issue: the ML model can generate test cases and error messages that are convincing to you and me regardless of whether they're actually right. Don't get me wrong, ML will reach a point someday where it catches up to humans in this respect; this is merely a shortcoming of today's ML, not tomorrow's, and it highlights where ML's weaknesses currently lie (in contrast to more…

It doesn't work in "closed book" form, the only way you can do that now is to have good tests and let the language models try many attempts.

Re: Study finds AI assistants help developers produce code likely to be buggy

#146

Post by Rastonbury »

klyrs wrote: post #106
johnhenning wrote: post #94

Looking at the demographics of the study, they only had 47 total participants, 15% of which were professionals and 62% reporting less than 5 years experience coding (which I would imagine is an underestimate assuming some people exaggerated their actual experience level). So that means they only had 6-7 participants who worked in industry and generously 18 people with more than 5 years experience. They also don’t rep…

My concern is that students/novices are going to be using this, without the ability to double-check the output of the tool. It inspires overconfidence, looks okay at the surface level, and bugs go unnoticed. The younger generation using this as a crutch, treating their own creations as a black box, will not have an adequate feedback mechanism to learn from their mistakes. Code quality and performance will deteriorate…

What is the current feedback mechanism and will they not use existing feedback mechanisms if available? Professionally someone should be there to enforce quality/mentor, but for students or hobbyists, even without AI assistants, they often don't have anyone to say "this is bad, this is best practise" except stackoverflow

Re: Study finds AI assistants help developers produce code likely to be buggy

#147

Post by ineedasername »

tmtvl wrote: post #124
ineedasername wrote: post #61

P¹ All people write buggy code. [1] P² AI assistants are trained on code written by people ∴ People using AI assistants will write buggy code [1] Some might object to this, either in their own case or that of some of their professional associates. I would not trust the judgement of these people.

I will object to P1: assumes facts not in evidence. And anyway there are two logical problems: 1. Not all people write code. 2. Code can be written without bugs. It may be provable that a statistically significant amount of code written by people is buggy, by linking to any overview of CVEs.

#1 is a specificity issue, though I think the meaning was clear, but "All people [who code] write buggy code" would be fine instead. Fixed

#2 Code can be written without bugs, but it isn't.

As for facts not in evidence, given the empirical nature of the proposition we're basically running up against the same problems that Hume has with inductive reasoning. Which is why I did not state P1 axiomatically and merely footnoted it to say I wouldn't trust the judgement of someone who believed otherwise.

Re: Study finds AI assistants help developers produce code likely to be buggy

#148

Post by enduku »

The code correctness issue is certainly a big problem -- it is simply not enough to get 90% correct. The real world problems often lie in the remaining 10% edge cases.

I have a very different take on how AI can come up with a correct by construction code, with not necessarily using a probabilistic model (Deep learning, for example). I have it written as a blogpost here[0]. The sketch of the idea is that any problem is a data problem, and an algorithm could be discovered, and new code could be generated by projecting it into topological space, finding the code there, and reducing the dimensions back in program space. It could well be a decent application of abstract algebra/algebraic topology to AI and code generation problems.

[0] https://densebit.com/posts/24

Re: Study finds AI assistants help developers produce code likely to be buggy

#149

Post by saurik »

visarga wrote: post #105
saurik wrote: post #83

Earlier quoted context omitted.

> If only there were a way that we could have foreseen that an AI trained to write code in part by looking at people who, self-admittedly, don’t know how to write code, and people who write code for others with minimal context (Stack Overflow), would produce buggy code. It is a case of GIGO. So, I'll claim the real issue is just that this generation of AI isn't able to "learn", it merely "trains": if I were alone in…

> With a human, garbage in can lead to something fully legitimate out! Because we get to see the error messages, fix and try again. You can try this on chatGPT - give it a task, run the code, probably fails, copy the error back, and let it fix is errors. After a few rounds it gets the result with much higher probability than when you allow it one single shot. A language model can write programs, and then we can run t…

> Because we get to see the error messages, fix and try again.

As I noted explicitly, a human will get better even with garbage input even without access to a computer. I also explicitly noted how we are able to learn from a single well-reasoned note.

I recommend you seriously evaluate how you yourself learn if you truly believe that you only learn things using active feedback from external sources of truth via trial runs.

Re: Study finds AI assistants help developers produce code likely to be buggy

#150

Post by rstuart4133 »

lumb63 wrote: post #21

Surprising. If only there were a way that we could have foreseen that an AI trained to write code in part by looking at people who, self-admittedly, don’t know how to write code, and people who write code for others with minimal context (Stack Overflow), would produce buggy code. It is a case of GIGO. Most developers do not learn much from Stack Overflow. Why do we expect AI to fare better? In my experience, one in t…

> Surprising.

For me it was the least surprising thing I've read in a while.

One thing that has become clear is AI's learn to cover the common cases very well, and the less common not at all well. So you get a lot of mistakes in the area's where there is not a lot of training data. Sometimes those mistakes don't matter. Creating art is a great example - how do you even define a mistake in art? (Perhaps it's not so difficult, as apparently AI output is recognisable but it takes an experienced eye to spot where it's gone wrong.) So is playing a game like Go - playing the 2nd best move occasionally doesn't matter. Apparently protein folding fits into the same class as getting the basic shape right matters a lot. Ditto for voice to text - humans compensate, and the same for translation. There are lots of places mistakes don't matter.

But some applications are very intolerant of mistakes. Driving a car appears to be one. Programming is probably one of the worst as a minor mistakes don't just degrade the program, they can destroy it in a way that's utterly non-obvious. It seems to me the current generation of AI's are always going to struggle in areas where mistakes, even minor ones, are costly.

Interestingly, one area of programming mistakes are tolerable is review. Therefore I'd predict programmers will find AI's reviewing code is a net positive.

Post reply on HN