Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study finds AI assistants help developers produce code likely to be buggy

theregister.com

Post reply on HN
Print view View original

41–50 of 153 posts

Re: Study finds AI assistants help developers produce code likely to be buggy

#41

Post by tjpnz »

xwdv wrote: post #12

I find the best use of coding AI is for shitty little inconsequential apps. I could imagine a startup founder using it to feel like they are saving a lot of money on engineering costs by using it for an MVP. You will likely never use coding AI for things like embedded systems, financial software, manufacturing, flight computers, operating systems, software for managing networked infrastructures, various domain specif…

Any industry where functional safety's a factor will be safe in the longterm. The rest I'm not so sure about..

Re: Study finds AI assistants help developers produce code likely to be buggy

#42

Post by dbish »

jrvarela56 wrote: post #6

The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just fine, my workflow went something like this: - Identify a component in a system you want to build - Give ChatGPT a list of bullets and ask it to write tests - Review tests - Run tests, give ChatGPT output of failing tests - Add the code to your codebase, run th…

There is also a context window problem where you can only pass in 4k tokens (including the prompt). This is not that large of a window for providing history and information of the entire workflow. How were you trying to solve that? Also, if you’re building something yourself, you can use davinci-003 directly through their APIs instead of trying to build on top of chatgpt

Re: Study finds AI assistants help developers produce code likely to be buggy

#43

Post by oneoff786 »

Any tool that helps people with worse skills produce a product will probably reduce the average quality if they wouldn’t have produced anything at all before. That’s not a super strong argument for not having them.

Like Jupyter Notebooks.

Re: Study finds AI assistants help developers produce code likely to be buggy

#44

Post by dmix »

jrvarela56 wrote: post #6

The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just fine, my workflow went something like this: - Identify a component in a system you want to build - Give ChatGPT a list of bullets and ask it to write tests - Review tests - Run tests, give ChatGPT output of failing tests - Add the code to your codebase, run th…

I wonder if you could build a dataset of UI components (React/HTML/CSS) with a simple description and taxonomy then feed it into an AI algo

Re: Study finds AI assistants help developers produce code likely to be buggy

#45

Post by dmix »

eloff wrote: post #38

I’ve been using GitHub copilot for six months now. It definitely produces a lot of code that looks right, but isn’t. I review anything it produces carefully. I’ve introduced one bug to production where I can say that was likely from copilot. Compared to the number of bugs I produced in that time, that’s nothing. It’s a productivity tool, not magic, and it’s not replacing me anytime soon. Like the auto complete in my…

What was the bug it produced?

Re: Study finds AI assistants help developers produce code likely to be buggy

#46

Post by kneebonian »

dmix wrote: post #37
lumb63 wrote: post #21

Surprising. If only there were a way that we could have foreseen that an AI trained to write code in part by looking at people who, self-admittedly, don’t know how to write code, and people who write code for others with minimal context (Stack Overflow), would produce buggy code. It is a case of GIGO. Most developers do not learn much from Stack Overflow. Why do we expect AI to fare better? In my experience, one in t…

> having an “AI Assistant” must lower programmers’ guards against buggy code Why would you assume that? If it’s buggy a couple times, if everyone talks about how buggy and unreliable it is, it can easily become common knowledge and common practice to triple check the output.

Then how much time are you actually saving if you have to review everything it produces. The bottle neck was never typing speed, at that point all the AI is allowing you to do is produce more buggy code more quickly.

Re: Study finds AI assistants help developers produce code likely to be buggy

#47

Post by dmix »

jerf wrote: post #23
torben-friis wrote: post #8

> The study involved 47 people with varying levels of experience, including undergraduate students, graduate students, and industry professionals. Honestly, for things like catching potential sql injections, “industry professional” should be the start. I won’t trust a student of any kind further than I can throw them, for topics like maintainability, security or readable code. Those are too far away from what’s usual…

Having "industry professionals" in this sort of study actually puts it in the top tier of studies. Most studies don't even have that, they're all undergrad based. (Sometimes people express frustration that we don't pay much attention to "the science" in the programming field, and my response is generally to tell such people to take a closer look at the "science" they're trying to wave around. Studies based on "a clas…

> one doesn't need to run some sort of enormous study to establish the point that these code assistants are not a magic gateway to quality code at any experience level

You just have to use it a couple times to figure this out. It’s pretty obvious what the limitations are and most programmers are smart enough to understand what it is and what it isn’t.

This is why I’m skeptical it will be a problem, it’s not being sold that way and after using it that will be obvious.

So anyone dumb enough to treat it like that and trust the output blind probably wasn’t a good programmer before. And if they keep doing it they don’t have proper incentive structures to not produce buggy code (senior devs, bosses, customers etc will notice that output is getting worse at a minimum when the product breaks or doing QA).

Re: Study finds AI assistants help developers produce code likely to be buggy

#49

Post by AlotOfReading »

dmix wrote: post #37
lumb63 wrote: post #21

Surprising. If only there were a way that we could have foreseen that an AI trained to write code in part by looking at people who, self-admittedly, don’t know how to write code, and people who write code for others with minimal context (Stack Overflow), would produce buggy code. It is a case of GIGO. Most developers do not learn much from Stack Overflow. Why do we expect AI to fare better? In my experience, one in t…

> having an “AI Assistant” must lower programmers’ guards against buggy code Why would you assume that? If it’s buggy a couple times, if everyone talks about how buggy and unreliable it is, it can easily become common knowledge and common practice to triple check the output.

1) verifying code is harder than writing it and

2) verifying code requires domain knowledge, which implies that the utility of these models is limited to things could write myself if I weren't too lazy. That's hugely constricting.

Re: Study finds AI assistants help developers produce code likely to be buggy

#50

Post by Jensson »

digitalsushi wrote: post #40
lumb63 wrote: post #21

Surprising. If only there were a way that we could have foreseen that an AI trained to write code in part by looking at people who, self-admittedly, don’t know how to write code, and people who write code for others with minimal context (Stack Overflow), would produce buggy code. It is a case of GIGO. Most developers do not learn much from Stack Overflow. Why do we expect AI to fare better? In my experience, one in t…

If I had a little robot riding in the passenger seat that could tell me whether to go left, straight, or right, and it was correct 90% of the time, I'd think that was pretty great. I'd get where I needed to be, even with a couple mishaps. ML code suggestions are the same thing to me. If I don't know where I am going, I can just ask it for suggestions. And it's probably going to be what I want. In both cases, I am ann…

> If I had a little robot riding in the passenger seat that could tell me whether to go left, straight, or right, and it was correct 90% of the time, I'd think that was pretty great. I'd get where I needed to be, even with a couple mishaps.

Pretty sure you wouldn't when it starts telling you to make illegal turns, or tell you to turn where there are no roads etc, that is the state of language models for code. You'd likely run over a person or get stopped by police at some point if you listen to it, then it is easier to just turn it off so you can focus on driving. A bad assistant is much worse than no assistant.

Edit: And these models are more like Teslas autopilot than an assistant giving you directions, since you have to oversee what it does instead of letting it tell you what to do. An autopilot that does the wrong thing 10% of the time is horrible.

Post reply on HN