Happily having access to GitHub copilot, it very often generates the code that I want. So it saves me from typing and also often saves checking Stack Overflow. I think the libraries/packages you use also play a big influence in how easy it is for copilot to create security flaws. Still, more training against security holes would be appreciated.
GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
111–120 of 161 posts
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#112Earlier quoted context omitted.
The problem with boilerplate code is, that it is mostly generated once (by the ide) and then slightly modified. More like a template. In the end you get a lot of meaningless code (the generated code), with some meaningful parts inside. But you can’t see anymore what was generated and what was added manually without deep analysis of the commit log. It is much better to generate code on the fly during build, so it does…
Not without trade-off though, I've been on both spectrum. Template generated code allows you to modify things if you know how, while generation on the flies will need a bunch of options, hooks, and worse string-based evaluation call to make it modifiable. So it's better for code with little to no modification, while boilerplate / template are better for things that will be modified.
> Template generated code allows you to modify things if you know how
That's exactly the problem. and because templates allow you to modify the code, the template creators do no work on generalizing it so it covers all the use cases. So, on practice, templates usually require that you modify it.
And now you have a huge codebase, mostly with the default text, but with some changes at random, and one of those changes is breaking it. Good luck finding it.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#113I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…
> Security starts with deep understanding. I wonder if the way we are approaching it is wrong. We are basically putting text though a deep learning black box. The model might have learned some abstractions, but all in all it is just playing word games and trying to guess the most likely continuation of a string. Maybe we should go into the other direction and base such an AI on a really massive ontology. Instead of u…
You're touching on the "Neat -vs- Scruffy" dichotomy in AI. (But it's not necessarily a dichotomy -- they can be combined!)
https://en.wikipedia.org/wiki/Neats_and_scruffies
From the "Scruffy" side, there's Charles Rich's classic work on "Programmer's Apprentice".
https://dspace.mit.edu/handle/1721.1/6054
https://dspace.mit.edu/bitstream/handle/1721.1/6054/AIM-1004...
>The Programmer's Apprentice Project: A Research Overview
>MIT AI Lab Memo No. 1004, November 1987.
>Rich, Charles; Waters, Richard C.
>Abstract: The goal of the Programmer's Apprentice project is to develop a theory of how expert programmers analyze, synthesize, modify, explain, specify, verify, and document programs. This research goal overlaps both artificial intelligence and software engineering. From the viewpoint of artificial intelligence, we have chosen programming as a domain in which to study fundamental issues of knowledge representation and reasoning. From the viewpoint of software engineering, we seek to automate the programming process by applying techniques from artificial intelligence.
https://dspace.mit.edu/handle/1721.1/41967
https://dspace.mit.edu/bitstream/handle/1721.1/41967/AI_WP_1...
>Plan Recognition in a Programmer's Apprentice. Ph.D. Thesis proposal.
>MIT AI Lab Working Paper 147, May 1977.
>Rich, Charles
>Abstract: Brief Statement of the Problem: Stated most generally, the proposed research is concerned with understanding and representing the teleological structure of engineered devices. More specifically, I propose to study the teleological structure of computer programs written in LISP which perform a wide range of non-numerical computations. The major theoretical goal of the research is to further develop a formal representation for teleological structure, called plans, which will facilitate both the abstract description of particular programs, and the compilation of a library of programming expertise in the domain of non-numerical computation. Adequacy of the theory will be demonstrated by implementing a system (to eventually become part of a LISP Programmer's Apprentice) which will be able to recognize various plans in LISP programs written by human programmers and thereby generate cogent explanations of how the programs work, including the detection of some programming errors.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#114It is important to remember that Copilot can improve. 40% is not a bad baseline, but one data point does not give us much info, we should wait and see the rate of improvement.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#115Earlier quoted context omitted.
> I feel like Copilot is the wrong direction to optimize development. This is mostly going to help people with already poor understanding of what they are doing create even more crap. Sometimes you don't need an expert to produce highly secure, highly optimized code. Have you seen the crap that people buy at Walmart? The furniture is not heirloom furniture, the food is not a 3-star artisanal experience. Have you boug…
> Sometimes you don't need an expert to produce highly secure, highly optimized code. This is one for the ages.
(Strictly) copying data between protobufs is pretty hard to mess up.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#116Half joking:
so far GitHub Copilot is more feasible as tool for humans doing code-coverage for its input code, "given enough eyeballs, all bugs are shallow" style. When a developer goes, "huh, Copilot generated insecure code, better report it to the original project it learned it from" - if only Copilot was able to link to the original project, it would all be great and useful.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#117…Compared to 60% of circumstances in the meat-based developer control group? :)
I love that we always use the average here for these justifications. We just slowly chip away and any and all excellence. 10x memes aside, we all know what it's like to work with a truly talented and productive engineer versus your everyday schmoe collecting a paycheck. It's a story as old as time, and yet here we are doing the exact big factory industrialization techniques other industries have done and that is commoditize the thing that made them exceptional and eliminating artisanship, uniqueness, and ultimately quality and character.
It's a tragedy of the commons of a sort.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#118Earlier quoted context omitted.
You would have to not host your code publicly either, right?
Merely hosting your code publicly seems like it wouldn't give GitHub the right to train AI models on it. You could even say it's against your terms of use. And to do it, they would have to go out of their way to find your repo on the web and clone it—unlikely. My impression (NOT A LAWYER) is that by hosting your code in a public repo on GitHub, you agree to their terms and give them the right to "read" your code incl…
I would think that training a NN falls squarely in fair use.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#119…Compared to 60% of circumstances in the meat-based developer control group? :)
I love that we always use the average here for these justifications. We just slowly chip away and any and all excellence. 10x memes aside, we all know what it's like to work with a truly talented and productive engineer versus your everyday schmoe collecting a paycheck. It's a story as old as time, and yet here we are doing the exact big factory industrialization techniques other industries have done and that is comm…
It may be a tragedy, but I fail to see why it is a tragedy of the commons? Which resource that is a available to all is being overused? High-paying dev jobs? Those are not a commons in the sense that tragedy of the commons implies because lower-quality devs don't stand to benefit by only taking a smaller part of the job.
Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment
#120Earlier quoted context omitted.
Not possible. Such licenses are founded upon copyright doctrine, and copyright doesn’t protect against learning, natural or machine. As it stands (and this can certainly change), legal consensus in general (regardless of jurisdiction) is that if you publish your code where they can reach it, they can use it.
So would it (theoretically) be legal to train on the JS files services like gmail.com serve to the client? What about decompiled output of proprietary software like certain files in Windows and macOS?
I would expect so, though given the limitations of decompilation (in the absence of debug info) I don't know how useful it would be