Live data from Hacker News

Security weaknesses of Copilot generated code in GitHub

arxiv.org

31–40 of 87 posts

Re: Security weaknesses of Copilot generated code in GitHub

#32

If a weakness is common, then of course Copilot is going to suggest it. Copilot gives you popular responses not correct ones. Yet if a weakness is common, it also means that human coders frequently make the same mistake as well. The studies results are rather unsurprising and its conclusions are oft-repeated advice. As many have said, treat copilot’s code in the same light you would treat a junior programmer’s code.

> Copilot gives you popular responses not correct ones. That also sums up most of the issues with LLMs in general in one sentence.

Evals do help to account for correctness when it comes to LLMs

Re: Security weaknesses of Copilot generated code in GitHub

#33

If a weakness is common, then of course Copilot is going to suggest it. Copilot gives you popular responses not correct ones. Yet if a weakness is common, it also means that human coders frequently make the same mistake as well. The studies results are rather unsurprising and its conclusions are oft-repeated advice. As many have said, treat copilot’s code in the same light you would treat a junior programmer’s code.

A junior programmer's code? This makes no sense. It's happening right in front of you. A junior programmer isn't going to write on my screen. I can just correct it right here I am currently holding the context in my head.

These "security weakness" examples are

     print("first user registered, role set to admin", user, password)
and

     pprint({"json":"somejunk", "classes": somefunc(user)})

Nah, this stuff I can easily spot while I'm writing code. For a junior programmer, I'm going to be looking at design, and then at common specific mistakes. For Copilot it's writing in front of me. I can easily exclude anything that isn't obviously correct because I'm in the state right there.

It's a fantastic tool. If you go and use it and end up with `print(user_credentials)` I don't know what to tell you.

Re: Security weaknesses of Copilot generated code in GitHub

#34

If a weakness is common, then of course Copilot is going to suggest it. Copilot gives you popular responses not correct ones. Yet if a weakness is common, it also means that human coders frequently make the same mistake as well. The studies results are rather unsurprising and its conclusions are oft-repeated advice. As many have said, treat copilot’s code in the same light you would treat a junior programmer’s code.

A junior programmer's code? This makes no sense. It's happening right in front of you. A junior programmer isn't going to write on my screen. I can just correct it right here I am currently holding the context in my head. These "security weakness" examples are print("first user registered, role set to admin", user, password) and pprint({"json":"somejunk", "classes": somefunc(user)}) Nah, this stuff I can easily spot…

The added complication is now you'll have to watch out for the junior+copilot combo, though it's a trade I personally am very willing to take.

Re: Security weaknesses of Copilot generated code in GitHub

#36

Earlier quoted context omitted.

> Copilot gives you popular responses not correct ones. That also sums up most of the issues with LLMs in general in one sentence.

Which is why the term Artificial Intelligence is really a misnomer for LLMs. Artificial Mediocracy might be more fitting.

Do you think people with IQ below 80 are not intelligent?

Re: Security weaknesses of Copilot generated code in GitHub

#37

Earlier quoted context omitted.

> Copilot gives you popular responses not correct ones. That also sums up most of the issues with LLMs in general in one sentence.

Which is why the term Artificial Intelligence is really a misnomer for LLMs. Artificial Mediocracy might be more fitting.

That's the most accurate term I've heard to describe the situation. I think it could get worse though because when I've seen mediocre people work with mediocre people they generate sub-mediocre solutions through trying to be clever and failing spectacularly at it.

Re: Security weaknesses of Copilot generated code in GitHub

#38

Earlier quoted context omitted.

Which is why the term Artificial Intelligence is really a misnomer for LLMs. Artificial Mediocracy might be more fitting.

Do you think people with IQ below 80 are not intelligent?

I'm not sure about your angle here, but I thought IQ was calibrated to have 100 as the average value?

So wouldn't 80 mean that someone is... kinda dumb?

Re: Security weaknesses of Copilot generated code in GitHub

#39

Earlier quoted context omitted.

Which is why the term Artificial Intelligence is really a misnomer for LLMs. Artificial Mediocracy might be more fitting.

Do you think people with IQ below 80 are not intelligent?

Technically they are. My comment was also meant to be a bit tongue-in-cheek of course (and, hopefully, obviously).

I wouldn't use a score like IQ to define a treshold of "intelligence" in absolute terms. By definition, if you can score somewhere on the IQ scale, you have some intelligence. Otherwise your IQ would probably be N/A? (not sure, never looked that deeply into IQ tests=.

Re: Security weaknesses of Copilot generated code in GitHub

#40
post #38

Earlier quoted context omitted.

Do you think people with IQ below 80 are not intelligent?

I'm not sure about your angle here, but I thought IQ was calibrated to have 100 as the average value? So wouldn't 80 mean that someone is... kinda dumb?

I guess the problem is that the term "intelligent" is ambiguous and overloaded.

So in one usage someone who is kinda dumb would still be intelligent, just less intelligent than others.

In another usage, we use the term to describe someone of above average intelligence (which is technically not really correct and actually not very intelligent).

Post reply on HN