Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

theinsaneapp.com

Post reply on HN
Print view View original

101–110 of 161 posts

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#101

Post by lhorie »

gverrilla wrote: post #55
lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

Do you think only experts should be programming? I'm an amateur programmer, and I think copilot could help me a lot with unimportant things, as you said - I even tried to install it but I'm not on some list. I can read code, and have built a few programs - I've hired around 30 different programmers in my life, and the vast majority clearly are copy-pasters-adapters. The way I see it, that happens because programming…

From my experience with tutoring, I would say that it won't help. The best way to learn is to mechanically do the work. Having things fed to you yield poorer results, IME.

Another parallel: language learning. You learn more by speaking and writing than merely reading and listening, because the former actually requires you to actively associate grammar rules to your physical actions, whereas consumption has a lower bar of effort since you can infer things from context, gloss over things, etc.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#102

Post by supernovae »

lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

I don't understand why people fear copilot or blame copilot.

Copilot doesn't bypass peer review, code review, unit testing so on and so forth.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#103

Post by lmilcin »

captainmuon wrote: post #99
lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

> Security starts with deep understanding. I wonder if the way we are approaching it is wrong. We are basically putting text though a deep learning black box. The model might have learned some abstractions, but all in all it is just playing word games and trying to guess the most likely continuation of a string. Maybe we should go into the other direction and base such an AI on a really massive ontology. Instead of u…

I think one way this could be improved is, instead of giving an exact answer (which is provably impossible to do correctly) maybe it could be possible to point the developer to other repositories where other people were solving similar problem.

There are already services that do this for you and I actually find them useful. For example, I might be trying to use a function from some library and it fails. If I get pointed to some public repositories that use the same library in function for similar purpose, I may learn that I am missing some critical setup. I can also browse different uses of this function/library and get informed on how it is at the very least used successfully by others.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#104

Post by [deleted] »

supernovae wrote: post #102
lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

I don't understand why people fear copilot or blame copilot. Copilot doesn't bypass peer review, code review, unit testing so on and so forth.

[deleted]

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#105

Post by macksd »

gverrilla wrote: post #55
lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

Do you think only experts should be programming? I'm an amateur programmer, and I think copilot could help me a lot with unimportant things, as you said - I even tried to install it but I'm not on some list. I can read code, and have built a few programs - I've hired around 30 different programmers in my life, and the vast majority clearly are copy-pasters-adapters. The way I see it, that happens because programming…

I sympathize with where you're coming from, but the phrase "unimportant things" bothers me. I'm always seeing clients deploy alpha or beta software in production. I see tech companies accumulating tech debt like nobodies business. None of that should happen. And often disasters involving tech get traced back to a cascading failure that started with something considered unimportant.

I love that software is an accessible discipline to hobbyists and that it empowers people. But it needs to be a discipline, top to bottom. We need deep understanding with security and robustness as fundamentals, good practices, and all of that baked into our tools.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#106

Post by simion314 »

andix wrote: post #96
simion314 wrote: post #93

Earlier quoted context omitted.

Code generated by magic is worse in my experience, with non-magic code you put a breakpoint where the project starts and you can run step trough it line by line, function by function and it makes sense. What I hate are magic frameworks that are terrible at reporting the issues, say in angular1 you have some bindings and soemtimes they don't trigger , you can't debug the magic strings of the templates in the debugger…

Generated code doesn't have to be magical. The generated code should off course be reviewed by a person from time to time. It must be debuggable and readable/understandable too. Otherwise it doesn't make any sense. A good example for generated code are typed clients for an OpenAPI interface. Instead of writing a REST client on your own based on a spec, you generate it. And if something isn't right in the first place,…

>Generated code doesn't have to be magical.

Sure, other good example is for example in Qt, the Designer tool will create some XML that shows the widget you placed properties, then a tool will generate code that is easy to read(not obfuscated or clever).

Can you give some examples on what kind of bad code generation/boilerplate you mean when you think at Java/C# ?

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#107

Post by harlekein »

Vaslo wrote: post #74

It's learning from existing code, right? Doesn't this say something about developers in general, or is the thought that it uses combinations of code that are insecure?

I don't hold the average developer in very high regard. There are tons of developers who are much better than me and I readily read their books, follow their tweets, blog posts and online talks to learn from them. I hold them in high regard, but these people are not the average developer.

If you would pick any smaller company with a dev team, a freelancer or an agency, your chances of finding a developer who understands and upholds quality code is vastly reduced.

Not to mention a lot of beginners will just push their practice projects to GitHub and never look at it again. I'm also guilty of this, but I never realized Microsoft was training AI with this code. If Copilot is learning from these projects then I'd say the code it regurgitates is not average, but even below average.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#108

Post by harlekein »

lampe3 wrote: post #32

I'm using copilot now for some time and yeah it's more a toy than real help right now. The only time it really helped when I needed to create a named list of char codes. When it comes to more complex code than checking the code of copilot takes the same time as writing it. 90% of the time I needed to correct copilot. For me, tools like linters are way more helpful then. If I could only use ESLint or copilot, I would…

I think another risk with getting Copilot to start out, is that it might nudge you into a direction you wouldn't have gone into otherwise.

Whether that is better or not, I suppose, it depends.

Re: GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment

#109

Post by amw-zero »

LeifCarrotson wrote: post #77
lmilcin wrote: post #2

I thought this should have been expected. Security starts with deep understanding. Some standards and practices can help avoid some types of problems, and some are even rather effective (like airgapping your systems), but there isn't any way to assure security in general other than truly understand what you are doing. ** I feel like Copilot is the wrong direction to optimize development. This is mostly going to help…

> I feel like Copilot is the wrong direction to optimize development. This is mostly going to help people with already poor understanding of what they are doing create even more crap. Sometimes you don't need an expert to produce highly secure, highly optimized code. Have you seen the crap that people buy at Walmart? The furniture is not heirloom furniture, the food is not a 3-star artisanal experience. Have you boug…

> Sometimes you don't need an expert to produce highly secure, highly optimized code.

This is one for the ages.

Post reply on HN