As always the statistic is useless without the human comparison. If it improves on human coders, no amount of gnashing and wailing will stop the layoffs.
Security weaknesses of Copilot generated code in GitHub
21–30 of 87 posts
Re: Security weaknesses of Copilot generated code in GitHub
#22If a weakness is common, then of course Copilot is going to suggest it. Copilot gives you popular responses not correct ones. Yet if a weakness is common, it also means that human coders frequently make the same mistake as well. The studies results are rather unsurprising and its conclusions are oft-repeated advice. As many have said, treat copilot’s code in the same light you would treat a junior programmer’s code.
> Copilot gives you popular responses not correct ones. That also sums up most of the issues with LLMs in general in one sentence.
With humans too
Re: Security weaknesses of Copilot generated code in GitHub
#23Re: Security weaknesses of Copilot generated code in GitHub
#24I wonder if it would be possible to rate the code used during the training phase. For example the code could go through various static analysis tools and the result would be assigned as metadata to the code being used to train the model. The final model would then know that a given pattern is flagged as problematic by some tool and could take this into account not just to suggest new snippets but also to suggest improvements of existing snippets. Though I suppose if it was that easy, they'd have done it already.
Re: Security weaknesses of Copilot generated code in GitHub
#25Earlier quoted context omitted.
That’s no longer true. Copilot uses the same ChatGPT-3.5 model as, well, ChatGPT. If it were trained on just GitHub projects, the chat features wouldn’t work at all.
You're assuming that Copilot Chat and the regular completion are the same model. Do you have a source that says so? I'd assumed that they were two different models, since they're quite different tasks.
Footnote 1 on page 2 explicitly mentions the 3.5 model and the research in this paper is only about auto completion: https://arxiv.org/pdf/2306.15033.pdf
And this blog post states “beyond Codex”, again for auto completion: https://github.blog/2023-07-28-smarter-more-efficient-coding...
Lastly, OpenAI states on the original Codex page: “OpenAI Codex is a descendant of GPT-3; its training data contains both natural language and billions of lines of source code from publicly available sources, including code in public GitHub repositories.” - It included GitHub repos, but it never was only GitHub repos. https://openai.com/blog/openai-codex
Update: GitHub Community Manager confirms it here: https://github.com/orgs/community/discussions/56975#discussi...
Re: Security weaknesses of Copilot generated code in GitHub
#26I wonder if it would be possible to rate the code used during the training phase. For example the code could go through various static analysis tools and the result would be assigned as metadata to the code being used to train the model. The final model would then know that a given pattern is flagged as problematic by some tool and could take this into account not just to suggest new snippets but also to suggest impr…
This is probably the next step for the LLM providers. They need to find ways to increase quality, and for code, there are many options. Perhaps code repos could get in on this too.
Re: Security weaknesses of Copilot generated code in GitHub
#27I wonder if it would be possible to rate the code used during the training phase. For example the code could go through various static analysis tools and the result would be assigned as metadata to the code being used to train the model. The final model would then know that a given pattern is flagged as problematic by some tool and could take this into account not just to suggest new snippets but also to suggest impr…
Wouldn't this train it to avoid detection more than to avoid bad patterns?
Re: Security weaknesses of Copilot generated code in GitHub
#28I wonder if it would be possible to rate the code used during the training phase. For example the code could go through various static analysis tools and the result would be assigned as metadata to the code being used to train the model. The final model would then know that a given pattern is flagged as problematic by some tool and could take this into account not just to suggest new snippets but also to suggest impr…
Wouldn't this train it to avoid detection more than to avoid bad patterns?
Yes, but presumably in the training data those two are quite correlated.
Re: Security weaknesses of Copilot generated code in GitHub
#29Earlier quoted context omitted.
> Copilot gives you popular responses not correct ones. That also sums up most of the issues with LLMs in general in one sentence.
Sums up the issues with democracy too, and a ton of other stuff
educating the "low-hanging fruit" is much more effective in moving the average than piling on excellence.
Re: Security weaknesses of Copilot generated code in GitHub
#30If a weakness is common, then of course Copilot is going to suggest it. Copilot gives you popular responses not correct ones. Yet if a weakness is common, it also means that human coders frequently make the same mistake as well. The studies results are rather unsurprising and its conclusions are oft-repeated advice. As many have said, treat copilot’s code in the same light you would treat a junior programmer’s code.
> Copilot gives you popular responses not correct ones. That also sums up most of the issues with LLMs in general in one sentence.
Which is why the term Artificial Intelligence is really a misnomer for LLMs. Artificial Mediocracy might be more fitting.