GitHub Copilot makes insecure code even less secure, Snyk says

Is this a surprise, though? I think any sane developer would expect that a generative AI code helper making suggestions based on existing patterns in your code would do exactly this. It's not "thinking" about security, performance, or other non-functionals.

Whenever I read articles like this (AI assistants having a negative impact on resulting work), I wonder about how much it affects "experienced" AI users. I've been interested in AI since the Cleverbot days, and have extensively used GitHub Copilot and ChatGPT since they came out. When I ask ChatGPT something that has an objective answer, but one I can't easily verify from my own knowledge or low stakes experiment (e.g. does this fix my syntax error?), I always make sure to not "ingest" it into my knowledge or product before finding one or more external corroborating sources. This doesn't make ChatGPT significantly less useful to me, from my experience, verifying an answer is typically much easier than researching the question from the ground up by conventional means (Google, GitHub Code Search). Similarly, when using GitHub Copilot, I am acutely aware that I need to critically evaluate the suggested code myself, and if there is something I am unsure about, it's again off to Google or Code Search.

Personally, the most risky AI stuff I do is if I am completely stuck on something, I might accept AI suggestions without much thought just to see if it can resolve whatever issue I am running into. But in my mind, those parts of the code are always "dirty" until I thoroughly review them; in the vast majority of cases, I end up refactoring those parts myself. If I am asking AI to improve a text I wrote, I rarely just take it as-is, I typically open both versions next to each other and apply parts I like to my original text.

In my opinion, stuff created by AI is inherently "unfinished". I cringe whenever people have AI do something and just roll with it (writing an essay, code, graphic design, etc.). AI is excellent for going most of the way, but in most cases, there need to be review and finishing touches by a human, at least for now.

It's worth nothing that over the holidays, I decided to pick up Go for a personal project and tried GitHub CoPilot. I was tremendously impressed. At the beginning, it was mostly useful for helping me with syntax as I could ask it to write a for loop, access member variables, etc.

Over time, it would start to suggest larger and larger blocks of code based on what I was writing. It got to the point where it would auto-suggest entire functions based on my code comments. e.g. I'd type: "//Update a user" and it would then suggest complete code for doing just that, while managing to match the style of previous methods, use the correct ORM objects, etc. In general, it seemed to be 90-95% "correct".

However, it did like to remind me that it was just really good at putting words together. For the Update user, it suggested a function that exactly matched how I'd write it, except for this:

//Persist changes to Show object to the database

user.Save()

Just a nice reminder that the GenAI is not "thinking" about what it's doing.