Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

theinsaneapp.com

Post reply on HN
Print view View original

151–160 of 161 posts

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#151

Post by phscguy »

amw-zero wrote: post #109
LeifCarrotson wrote: post #77

Earlier quoted context omitted.

> I feel like Copilot is the wrong direction to optimize development. This is mostly going to help people with already poor understanding of what they are doing create even more crap. Sometimes you don't need an expert to produce highly secure, highly optimized code. Have you seen the crap that people buy at Walmart? The furniture is not heirloom furniture, the food is not a 3-star artisanal experience. Have you boug…

> Sometimes you don't need an expert to produce highly secure, highly optimized code. This is one for the ages.

I think they meant that sometimes you don't need secure and optimized code and hence you don't need an expert.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#153

Post by perl4ever »

captainmuon wrote: post #99
lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

> Security starts with deep understanding. I wonder if the way we are approaching it is wrong. We are basically putting text though a deep learning black box. The model might have learned some abstractions, but all in all it is just playing word games and trying to guess the most likely continuation of a string. Maybe we should go into the other direction and base such an AI on a really massive ontology. Instead of u…

There was a project started in 1984 to do that:

https://en.wikipedia.org/wiki/Cyc

Supposedly an attempt to assemble a database of "common sense" facts and reasoning.

It has always been controversial and it's not clear what kind of success it's had.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#154

Post by chrismorgan »

JamesSwift wrote: post #80
chrismorgan wrote: post #43

Earlier quoted context omitted.

No, the license that you apply is completely irrelevant, and there’s certainly nothing whatsoever special about the GPL. Copilot is completely depending on being effectively exempt from copyright; if that legal theory falls apart, the entire space (and a lot of other machine learning stuff) is utterly doomed. Trouble is, Copilot can’t tell whether it’s reproducing copyrightable chunks of your code, or indeed where wh…

They could easily tag the source with license info and take that information into account when feeding data in.

That’s not how learning, human or machine, works. Learning is about collecting all kinds of stuff from diverse sources into a great melting-pot, so that you can form something new out of it—but you can’t generally identify where everything comes from. Individual recognisable tricks perhaps, but if you want to say “this code was inspired by X, Y and Z”, well, that inspiration is typically everything, the entire corpus.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#155

Post by Barrin92 »

LeifCarrotson wrote: post #77
lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

> I feel like Copilot is the wrong direction to optimize development. This is mostly going to help people with already poor understanding of what they are doing create even more crap. Sometimes you don't need an expert to produce highly secure, highly optimized code. Have you seen the crap that people buy at Walmart? The furniture is not heirloom furniture, the food is not a 3-star artisanal experience. Have you boug…

>Have you seen the crap that people buy at Walmart? The furniture is not heirloom furniture,

the difference is that your sofa isn't programmable and networked into every other appliance in your house underpinned by a general purpose computer rife for abuse.

Virtually every piece of software you install is an access point to your machine or your sensitive data. One isolated thing in the analog world breaks down, not a problem. One misconfigured password in a VPN client, and whoops part of your national oil infrastructure goes offline

https://www.reuters.com/business/colonial-pipeline-ceo-tells...

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#156

Post by nextlevelwizard »

0-_-0 wrote: post #85
nextlevelwizard wrote: post #64

Earlier quoted context omitted.

It is useful since it means copilot is not taking your job any time soon. i.e. if 40% of the time the human driving the thing is needed to intervene and prevent obvious security flaws then expert is still needed to use the tool.

I think it was obvious from the beginning that it's trained on GitHub code, so it would be surprising if it was better than the average code on GitHub. In any case, if Copilot can generate code as well as the average programmer without supervision, that means it can already take the job of 50% of programmers. A more useful metric though is how many programmers can a person using Copilot replace by having greater prod…

With any commercial software security matters. You don't want your licensing solution to be craked by editing a file.

Anything that connects to any server or has any sort of networking.

Anything that needs privileges (like installing drivers) needs security.

Anything that reads/interpeds any data given to it needs to have security.

Of course this all depends what you mean by "matter". I can't come up with a program that doesn't need to think about security at all expect something trivial like hello world.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#157

Post by nextlevelwizard »

tyingq wrote: post #68
nextlevelwizard wrote: post #62

Earlier quoted context omitted.

You can always host is with license that doesn't allow reuse or something

GitHub mentions that they don't currently look at the license before trawling code. https://twitter.com/NoraDotCodes/status/1412741339771461635 There's also other references that GitHub public repos weren't the only source. They trawl other publicly readable code.

You can sue them for using your code if they break the licensing agreement. Contact EFF and they'll set you up with a lawyer.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#158

Post by moonchrome »

discreteevent wrote: post #98
moonchrome wrote: post #49

Earlier quoted context omitted.

>Java is not the culprit here. It definitely is the culprit. They didn't even want to add `var` to the language until recently, and let's not even go to the anonymous class vs lambdas retardation. These are just the things that they eventually buckled on, but Java is extremely boilerplatey - the bad patterns and XML crap got invented to deal with that problem. DDD and onion are another issue, mostly coming out of the…

The TDD movement came from smalltalk programmers (not that I think it has anything to do with smalltalk, just the programmers came from there). In my experience it was in Ruby and javascript code where I have seen the most inane micro(nano?)-unit tests. Some of this was partly because the unit tests were testing what a static typechecker could verify automatically (at the cost of a little verbosity). I don't see how…

Maybe I shouldn't call it TDD camp, but the people that were prothletising TDD were also selling onion/DDD in C# world, so I just bunch those together, it's more about onion, layered architectures, testing at different layers and mocking them, etc.

While I consider TDD the wrong approach in 90% of scenarios, in dynamic languages it works out much better because the object model is so flexible you can mock just about anything trivially. In C# and Java it's just boilerplate on top of boilerplate.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#159

Post by amw-zero »

phscguy wrote: post #151
amw-zero wrote: post #109

Earlier quoted context omitted.

> Sometimes you don't need an expert to produce highly secure, highly optimized code. This is one for the ages.

I think they meant that sometimes you don't need secure and optimized code and hence you don't need an expert.

That is definitely not a possible interpretation:

> to produce highly secure, highly optimized code.

The key word there is "produce" - meaning, the secure and optimized code is being written.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#160

Post by cannabis_sam »

moonchrome wrote: post #49
lmilcin wrote: post #20

Earlier quoted context omitted.

(I have been professionally programming Java backends for the past 16 years). Java is not the culprit here. I think it is something that happened on the way that has something to do with J2EE and patterns craze we had a decade ago or two ago. It doesn't help that frameworks like Spring and their documentation go out of their way to propagate these boilerplate-heavy patters. Copying these lazy patterns is shortest, ea…

>Java is not the culprit here. It definitely is the culprit. They didn't even want to add `var` to the language until recently, and let's not even go to the anonymous class vs lambdas retardation. These are just the things that they eventually buckled on, but Java is extremely boilerplatey - the bad patterns and XML crap got invented to deal with that problem. DDD and onion are another issue, mostly coming out of the…

>They didn't even want to add `var` to the language until recently, and let's not even go to the anonymous class vs lambdas retardation.

This realization is always endlessly infuriating to me as someone who was taught way to much Java in Uni, and had to intensionally push myself into other languages to realize why simple things like higher-order functions were subjugated under the tyranny of classes in java.

But far worse than that is the absolutely abysmal and destructive philosophy around types in java. Just mash em together with namespaces and classes, and then nerf type inference to the point that it couldn’t infer what is literally the most trivial reflexion-based equality: “Object o = New Object()”.

Post reply on HN