Study finds AI assistants help developers produce code likely to be buggy
1–10 of 153 posts
Re: Study finds AI assistants help developers produce code likely to be buggy
#2I've been producing buggy code for over 20 years. Don't need no AI to help me do that.
Re: Study finds AI assistants help developers produce code likely to be buggy
#3Sounds like we need computer languages that are better at preventing bugs. Throw in a little static analysis and AI assistants become more useful.
Type inference, pure functions, functional, assertions, …
Maybe start with Haskell?
Re: Study finds AI assistants help developers produce code likely to be buggy
#4I made a site using the OpenAI APIs new models (similar to ChatGPT) for creating simple apps or demos using natural language: https://aidev.codes . If anyone is interested please try it out.
I tried to promote the site by submitting to HN and did not receive a single vote. I feel that although many people may not be impressed, it is better than 0. So I am just trying to promote it in related threads.
Anyway if you have any feedback or anything I am in the ChatGPT and OpenAI Discords as user runvnc.
Re: Study finds AI assistants help developers produce code likely to be buggy
#5Me:
When you help coders, does that make their code more buggy? Please give me a dramatic, agitated answer.
ChatGPT:
No, it does not make their code more buggy. In fact, my goal is to help coders write better, more reliable code by providing guidance and assistance. I am here to provide support and guidance, not to cause problems or introduce bugs into their code. As a language model, I do not have the ability to execute code or interact with the real world, so I cannot cause bugs or issues in a coder's code. My role is simply to provide information and guidance to help coders troubleshoot issues and write better code. So no, I do not make code more buggy – I strive to make it better!
Re: Study finds AI assistants help developers produce code likely to be buggy
#6The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just fine, my workflow went something like this:
- Identify a component in a system you want to build
- Give ChatGPT a list of bullets and ask it to write tests
- Review tests
- Run tests, give ChatGPT output of failing tests
- Add the code to your codebase, run the tests...
I developed extensions to my console and test suite so the output exceptions/failures ended up in my clipboard. I'm highly confident workflows that put the AI in a feedback loop will be able to code most backend stuff. UIs are trickier as giving it context is not trivial.
Re: Study finds AI assistants help developers produce code likely to be buggy
#7Sounds like we need computer languages that are better at preventing bugs. Throw in a little static analysis and AI assistants become more useful. Type inference, pure functions, functional, assertions, … Maybe start with Haskell?
Haskell? Not buggy code? I'd say Haskell is the best language to write buggy code. And when I say buggy is not about writing purposely to be buggy, I say it's a maintenance horror show that WILL produce buggy code 100% down the road. Remember, maintenance is 90% of a project life, the initial development is only 10%.
Re: Study finds AI assistants help developers produce code likely to be buggy
#8> The study involved 47 people with varying levels of experience, including undergraduate students, graduate students, and industry professionals.
Honestly, for things like catching potential sql injections, “industry professional” should be the start.
I won’t trust a student of any kind further than I can throw them, for topics like maintainability, security or readable code. Those are too far away from what’s usually taught at school.
Re: Study finds AI assistants help developers produce code likely to be buggy
#9I've been producing buggy code for over 20 years. Don't need no AI to help me do that.
But what if you could write more LoC/h and make Elon Musk happier?
Re: Study finds AI assistants help developers produce code likely to be buggy
#10Fred Brooks would surely have added AI assistants to the arsenal of supposed silver bullets that failed.
One specific problem with chatGPT as it is today is that it gives answers that sound extremely confident. So confident it's tempting to follow them without any critical thinking. Sure you can guide it towards the correct answer, but it requires you knowing the answer in advance, which defeats the whole point.