What’s the baseline? 60% may still be superior to the average implementation.
GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
31–40 of 161 posts
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#32I'm using copilot now for some time and yeah it's more a toy than real help right now.
The only time it really helped when I needed to create a named list of char codes.
When it comes to more complex code than checking the code of copilot takes the same time as writing it. 90% of the time I needed to correct copilot.
For me, tools like linters are way more helpful then. If I could only use ESLint or copilot, I would go 100% of the time with ESLint.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#33Meaning 40% of the code on GitHub is insecure
You could train a model on purely secure code and still have it combine it in insecure ways.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#34I fail to see how this is particularly useful information about Copilot. The comparison should be:
1. How many times do people write insecure code when not using Copilot?
2. How many times do people write insecure code when using Copilot?
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#35Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#36You are the free labor copilot to train Microsoft GitHub's Copilot tool. You are responsible for any of those insecure code errors and the diligence require. You will be on the hook for resulting problems. But Microsoft and their home-phoning, tracking-embedded editor will get real people to correct and train their machine for free—with their stated plan of later selling that machine back to us later. I wish there we…
It is called LICENSE.txt. License your code as GPL and then Copilot can't reproduce bigger parts of your code.
But as long as you give the public access to your code, they can study it and learn from it. Humans and machines.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#37I wonder if the Copilot model could somehow be repurposed to analyze the quality of a developer’s code. Seeing how Microsoft owns both GitHub and LinkedIn, it’s a good bet this is something they’re actively researching.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#38You are the free labor copilot to train Microsoft GitHub's Copilot tool. You are responsible for any of those insecure code errors and the diligence require. You will be on the hook for resulting problems. But Microsoft and their home-phoning, tracking-embedded editor will get real people to correct and train their machine for free—with their stated plan of later selling that machine back to us later. I wish there we…
It is called LICENSE.txt. License your code as GPL and then Copilot can't reproduce bigger parts of your code. But as long as you give the public access to your code, they can study it and learn from it. Humans and machines.
I don't think that the GPL gives much more protection than any other FOSS license, here, in practice.
If Copilot were to reproduce a larger part of, say, an MIT-licensed codebase or almost any other permissive licence, then they should legally provide attribution. I'm pretty sure that they don't even have an option to provide such specific attribution, which means that either they believe that the code copied from any one source is below the relevant threshold or they're just ignoring copyright.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#39Earlier quoted context omitted.
(I have been professionally programming Java backends for the past 16 years). Java is not the culprit here. I think it is something that happened on the way that has something to do with J2EE and patterns craze we had a decade ago or two ago. It doesn't help that frameworks like Spring and their documentation go out of their way to propagate these boilerplate-heavy patters. Copying these lazy patterns is shortest, ea…
It's not the fault of the language or the runtime. But some common Java/.NET frameworks nudge you into this direction. Also OOP is very commonly abused in those languages, to make easy stuff more complicated.
Just because objects are involved doesn't mean it is OOP code... OOP is completely misunderstood especially in Java community to the point where it is pretty difficult to see actually object oriented code.
A "service" with a bunch of stateless functions (I am intentionally not calling them methods) is really just a library of routines and the class is used mostly for namespace purposes (to group related functions together) and maybe deliver access to some dependencies. But those dependencies could be thought about almost the same way as global variables in a C program, because usually there exists only single instance of the service.
Neither are DTOs being passed between these services an OOP meachanism -- they are almost C-like structs to make it easier to pass data between functions and to have single reference to them. The only exception maybe is things like equals(), hashcode() etc, but this is very shallow use of OOP patterns.
So it is really difficult to say this is abuse of OOP, when there is very little of actual OOP in it.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#40You are the free labor copilot to train Microsoft GitHub's Copilot tool. You are responsible for any of those insecure code errors and the diligence require. You will be on the hook for resulting problems. But Microsoft and their home-phoning, tracking-embedded editor will get real people to correct and train their machine for free—with their stated plan of later selling that machine back to us later. I wish there we…
It is called LICENSE.txt. License your code as GPL and then Copilot can't reproduce bigger parts of your code. But as long as you give the public access to your code, they can study it and learn from it. Humans and machines.
I'd love to see ML-GPL which specifically deals with using licensed property as a training set.