GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

I thought this should have been expected.

Security starts with deep understanding.

Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing.

**

I feel like Copilot is the wrong direction to optimize development. This is mostly going to help people with already poor understanding of what they are doing create even more crap.

For a good developer those low level, low engagement activities are not a problem (except maybe for learning stage where you actually want people engaged rather than copy/paste). What it does not help is the important parts of development -- defining domain of your problem, design good APIs and abstractions, understanding how everything works and fits together, understanding what your client needs, etc.

Also, I feel this is going to help increase complexity by making more copies of same structures throughout the codebase.

My working theory about this is this is going to hinder new developers even more than they already are by google and stack*. Every time you are giving new developers an easier way to copy paste code without understanding you are robbing them an opportunity to gain deeper understanding of what they are doing and in effect prevent them from learning and growing.

It is a little bit like giving answers to your kids homework without giving them chance to arrive at the answer or explaining anything about it.

**

Another way I feel this is going to hurt developers is competition in who can produce most volume of code.

I have already noticed this trend where developers (especially more junior but aspiring to advance) try to outcompete others by producing more code, close more tickets, etc. Right now it means skipping understanding of what is going on in favor of getting easy answers from the Internet.

These guys can produce huge amounts of code with relatively little actual engagement.

To management (especially with wrong incentives) this seems like a perfect worker, because management usually doesn't understand the connection between lack of engagement and planning at design/development time with their later problems (or they don't feel it is them that is going to pay the price).

The Copilot is probably going to make it even more difficult for people who want to do it the right way because even starker difference in false productivity measurements.

For comparison, what percentage of human-generated code is secure?

I too feel this is the wrong direction, from the fundamental aspect that trained algorithms contain no comprehension of what they are doing. They are the classic idiot savant. In a technological economy, comprehension of the environment is everything. I do not see how comprehension can be achieved without the elusive General AI, so I do not see this as anything other than a new area of research exposing how vitally important it is to have comprehension.

> For comparison, what percentage of human-generated code is secure?

Yeah how did they measure? Did static and dynamic analysis find design bugs too?

Maybe - as part of a Copilot-assisted DevSecOps workflow involving static and dynamic analysis run by GitHub Actions CI - create Issues with CWE "Common Weakness Enumeration" URLs from e.g. the CWE Top 25 in order to train the team, and Pull Requests to fix each issue?: https://cwe.mitre.org/top25/

Which bots send PRs?

You are the free labor copilot to train Microsoft GitHub's Copilot tool. You are responsible for any of those insecure code errors and the diligence require. You will be on the hook for resulting problems. But Microsoft and their home-phoning, tracking-embedded editor will get real people to correct and train their machine for free—with their stated plan of later selling that machine back to us later.

I wish there were a “robots.txt” file for Git to disallow certain bots from training on anything I have written.