Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Good and the Limitations of Github Copilot

blog.hrithwik.me

Post reply on HN
Print view View original

61–70 of 144 posts

Re: The Good and the Limitations of Github Copilot

#61

Post by philderbeast »

tsimionescu wrote: post #11
visarga wrote: post #10

Earlier quoted context omitted.

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

> Also, it has the same copyright problems as if you used Google to find the code. No, it has the same copyright problems as if you Google and instead of getting links to sites that host code and licenses, you get just the code.

It's even more gray then that.

Regardless of the licence, does the produced code even quality for copywrite protection, or does it fall under fair use?

what licence, if any, is there for unique code generated by co-pilot etc etc etc.

its a great big ball of who knows, however I expect that noting your only getting snippets you would be highly unlikly to get code that dosent fall under the fair use provisions, that said IANAL

Re: The Good and the Limitations of Github Copilot

#62

Post by busymom0 »

visarga wrote: post #10
chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

> same copyright problems

Is this true? From what I remember reading, the code was uniquely created (but I could be wrong). If that's the case, then does it tel you what license the generated code is under?

Re: The Good and the Limitations of Github Copilot

#64

Post by objclxt »

visarga wrote: post #10
chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

> Copilot acts like a search engine, you search, you find, then you judge.

OK, but that’s not how GitHub position it:

“Your AI pair programmer” “Skip the docs and searching for examples”

They literally say it’s not a search engine!

Re: The Good and the Limitations of Github Copilot

#67

Post by chrismorgan »

bamboozled wrote: post #53
avar wrote: post #52

Earlier quoted context omitted.

> It generates a nastily complex regular expression that is hopelessly wrong. > [...] > The author makes no comment on how hideously bad it is[...] I mean, it's coming up with a solution that's about as good as the average programmer who's going to validate E-Mail with regexes would, so as a crowd-sourced machine learning solution it's not too bad if you think about it. In other words, Having a co-pilot doesn't mean…

Here’s the difference imo, an average programmer with some experience would seek out well tested and used library to help them do something like this, not use sausage meet spat out of a cannon to validate email addresses.

I’d hope so, but then again, this is what their ML model spat out, which suggests people have written stuff like this, though hopefully not the wonkiness around the bit after the last dot.

But on the brighter side, the material the user provided to Copilot in this case was pretty much “I want to implement email validation from scratch” rather than “I want to validate an email address”, which is where hopefully people would look more to existing libraries. And they’ll commonly already such libraries or functions in their code base, e.g. under Django you’d use… uh oh, searching found https://stackoverflow.com/q/3217682/ first which looks frighteningly familiar here in half of the errors it contains; but anyway, you should use https://docs.djangoproject.com/en/3.2/ref/validators/#emailv.... I suspect the Copilot approach as used will be unintentionally biased much more towards boilerplate and implementing things from scratch, rather than using libraries.

Re: The Good and the Limitations of Github Copilot

#68

Post by yyyk »

avar wrote: post #52
chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

> It generates a nastily complex regular expression that is hopelessly wrong. > [...] > The author makes no comment on how hideously bad it is[...] I mean, it's coming up with a solution that's about as good as the average programmer who's going to validate E-Mail with regexes would, so as a crowd-sourced machine learning solution it's not too bad if you think about it. In other words, Having a co-pilot doesn't mean…

>about as good as the average programmer who's going to validate E-Mail with regexes would

IMHO, the average programmer is not even aware of regexs, which is a problem, but here would lead the programmer to a simpler to read solution.

You need to be at a very particular point (good enough to be proficient with regexs, but bad enough to use them for everything and bad enough to not test your regex), and I strongly doubt that's the average.

P.S. The video question was to validate emails, not 'validate using regex'.

Re: The Good and the Limitations of Github Copilot

#69

Post by ZiiS »

omk wrote: post #28
lilyball wrote: post #25

Earlier quoted context omitted.

This is exactly the problem. The regex issue isn’t that it’s not efficient, it’s that it’s wrong. Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous.

> Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous. I would like this message to be amplified as much as possible. Never write code you do not understand. I am excited about copilot, but also wary of the programming culture these tools will bring in. Businesses, especially body-shopping companies will want to deliver as much using tools in thi…

Isn't "Code you don't understand" the definition of AI/ML?

Re: The Good and the Limitations of Github Copilot

#70

Post by BigJono »

tlrobinson wrote: post #15
nicetryguy wrote: post #3

Is === really necessary for Javascript? Type checking seems like overkill?

It’s a well known rule in JavaScript to default to “===“ for everything except checking for null or undefined, in which case “== null” is acceptable. https://eslint.org/docs/rules/eqeqeq#smart Even better is to just use TypeScript.

Checking for 'null or undefined' is an antipattern anyway, so there's no real valid use of '=='.
Post reply on HN