The Good and the Limitations of Github Copilot • Hacker News Board

Re: The Good and the Limitations of Github Copilot

#141

Post by shadowgovt » Mon, Jul 05, 2021, 5:51 PM UTC

cf499 wrote: post #132

shadowgovt wrote: post #128

Earlier quoted context omitted.

When I'm that resource constrained, I don't use Python. The changes necessary to make regular Python code performant defeat the goal of making it readable. There's a reason NumPy's innards aren't Python code.

Both sides of this "readability vs. performance" debate can be argued ad absurdum but that's not my intention. All I know is that I try to conserve resources no matter what level of the stack I'm working on and those generated snippets certainly don't!

Indeed, but they're not intended to. Nothing about Copilot's specs even claims it's generating optimally-performant code. And the Python example indicates why that may not even be a desirable output (most performant and most readable are orthogonal metrics).

Re: The Good and the Limitations of Github Copilot

#142

Post by jeffnappi » Tue, Jul 06, 2021, 1:13 AM UTC

chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Proper email validation: Contains '@' and a message sent to the address is delivered.

Re: The Good and the Limitations of Github Copilot

#143

Post by thrwyoilarticle » Tue, Jul 06, 2021, 12:33 PM UTC

bellyfullofbac wrote: post #58

Skimmed through this "review", nothing wowing me. > Hey Stephen, If you are reading this, please follow me on hashnode . God, that's obnoxious...

That's what we call a 'joke'

Re: The Good and the Limitations of Github Copilot

#144

Post by lilyball » Wed, Jul 07, 2021, 12:42 AM UTC

victor106 wrote: post #121

lilyball wrote: post #25

Earlier quoted context omitted.

This is exactly the problem. The regex issue isn’t that it’s not efficient, it’s that it’s wrong. Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous.

Very well put. Thank You. I know copilot is in alpha and will improve 100x but you will still need someone qualified to double check

I don't think it's possible for copilot to improve on this problem. It doesn't actually understand the code, it's just statistical models all the way down. There's no way for copilot to judge how good code is, only how frequently it's seen similar code. And frequency is not the same thing as quality.