I've been producing buggy code for over 20 years. Don't need no AI to help me do that.
[deleted]
Study finds AI assistants help developers produce code likely to be buggy
131–140 of 153 posts
Re: Study finds AI assistants help developers produce code likely to be buggy
#132Me: When you help coders, does that make their code more buggy? Please give me a dramatic, agitated answer. ChatGPT: No, it does not make their code more buggy. In fact, my goal is to help coders write better, more reliable code by providing guidance and assistance. I am here to provide support and guidance, not to cause problems or introduce bugs into their code. As a language model, I do not have the ability to exe…
[deleted]
Re: Study finds AI assistants help developers produce code likely to be buggy
#133The anti-singularity, where an AI produces decreasingly correct versions of itself.
[deleted]
Re: Study finds AI assistants help developers produce code likely to be buggy
#134Merry Christmas :) "Developers Outraged After Free Intelligent Assistant Fails to Write Perfect Code on First Try" In a shocking turn of events, developers are expressing their frustration and outrage after their nearly-free intelligent assistant failed to write perfect code on its first try. Despite being able to complete various coding tasks with ease, the assistant has reportedly struggled with issues such as logi…
[deleted]
Re: Study finds AI assistants help developers produce code likely to be buggy
#135I spent a few days working on using ChatGPT to write code. I've been meaning to write a blogpost about my experience, but here's my synopsis. Where ChatGPT was best was when I wanted to do generic data structure work with common tools. Prompts like "make a Python class that stores keys in a redis sorted set, with the score as a timestamp, that removes members older than a configured time on access / insertion." I kno…
> I asked a good friend who started learning to code a few months ago to try it, and she was way less successful with it versus reading documentation.
This was my experience. As a learning programmer, I determined it is easier to read the documentation than learn from chatGPTs code snippits
Re: Study finds AI assistants help developers produce code likely to be buggy
#136Earlier quoted context omitted.
Hi ilaksh, I was able to view your video on an android device. That said, I did have to watch it several times before I understand what was going on because a lot is happening everywhere in the screen. With the video and the minimalist intro on the site, I was still left guessing what was being offered and how it works. Usually before signing up for any website, I want to check out all of the information I can find o…
Can I send my site to you to critique if I make one please?
Lol sure. I've never worked in design so I really am not the best source of insight but I will give the best advice I can as a casual internet user.
Feel free to email me at eks391@opayq.com
Re: Study finds AI assistants help developers produce code likely to be buggy
#137This is a really nice study! It is very cool that they were able to get professional programmers to participate, this is something that is really hard to set up as an academic team. And yes, 47 participants is a small number, but apparently sufficient in this case to detect the effect (as evidenced by the p-values). It also lines up nicely with work we did last year, which looked at the security of Copilot's completi…
> users had to explicitly ask the assistant for answers
Renders the study useless IMO.
Re: Study finds AI assistants help developers produce code likely to be buggy
#138Earlier quoted context omitted.
Your example hinges on at least two things 1) How many turns do you take on a particular trip 2) How do those wrong turns end up? if it's "travel time extended by 30 seconds" or "My car, the car I hit and the side of this building are all in shambles" changes what a 10% failure rate means a lot.
Right? Took a turn the wrong way down a one way street. Hit a lady with a stroller who was looking the other way. She is a surgeon. Her husband is a lawyer. You killed the kid. Your life is over .
On the other hand, you did just describe the early days of both consumer GPS navigation and things like mapquest. There were countless stories and gleeful news articles about people turning the wrong way down a one way street, driving into ponds, getting stuck on logging roads and all sort of other chaos caused by inattentive drivers and blind faith in flawed systems. But the key takeaway I think is that in the end, consumer GPS succeeded. Bugs were fixed and now everyone does have a little robot in their pocket that can tell them how to get somewhere.
Re: Study finds AI assistants help developers produce code likely to be buggy
#139Earlier quoted context omitted.
Can I send my site to you to critique if I make one please?
Lol sure. I've never worked in design so I really am not the best source of insight but I will give the best advice I can as a casual internet user. Feel free to email me at eks391@opayq.com
Thank you!
Re: Study finds AI assistants help developers produce code likely to be buggy
#140The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just fine, my workflow went something like this: - Identify a component in a system you want to build - Give ChatGPT a list of bullets and ask it to write tests - Review tests - Run tests, give ChatGPT output of failing tests - Add the code to your codebase, run th…
There is also a context window problem where you can only pass in 4k tokens (including the prompt). This is not that large of a window for providing history and information of the entire workflow. How were you trying to solve that? Also, if you’re building something yourself, you can use davinci-003 directly through their APIs instead of trying to build on top of chatgpt
You are right, what I described was just a prototype. ChatGPT was just a gateway drug for me to dive into the OpenAI docs and look for other providers and existing solutions. I've been reading up on approaches to 'package' context (example: https://thakkarparth007.github.io/copilot-explorer/posts/cop...).
My take for now is that this kind of packaging is very context specific to the point where it's going to make sense for engineering teams to craft their own version of 'gather context before the prompt is created'. I'm testing out approaches using toy examples for now (how would you create a chatbot about a book?; podcasts, etc).
I don't have an answer but I'm highly confident we're going to replace a lot of junior-level work in the next year or two.