The Good and the Limitations of Github Copilot • Hacker News Board

Re: The Good and the Limitations of Github Copilot

#11

Post by tsimionescu » Mon, Jul 05, 2021, 5:09 AM UTC

visarga wrote: post #10

chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

> Also, it has the same copyright problems as if you used Google to find the code.

No, it has the same copyright problems as if you Google and instead of getting links to sites that host code and licenses, you get just the code.

Re: The Good and the Limitations of Github Copilot

#12

Post by foepys » Mon, Jul 05, 2021, 5:10 AM UTC

visarga wrote: post #10

chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

Does GitHub Copilot tell me which license the code it suggested has? If not it's a huge difference to code search engines.

Re: The Good and the Limitations of Github Copilot

#13

Post by dcow » Mon, Jul 05, 2021, 5:12 AM UTC

visarga wrote: post #10

chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

So if we classify AI as search.. and then claim fair use.. we can launder dirty viral code.. how could this go wrong?

But really, if thispersondoesnotexist is just a really good per-pixel search against a corpus of human faces where each “page” of result pixels is organized in a grid presented as a new image with its own metadata..

I mean I guess Google really was an AI company all along.

Re: The Good and the Limitations of Github Copilot

#14

Post by chrismorgan » Mon, Jul 05, 2021, 5:12 AM UTC

visarga wrote: post #10

chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

Nice theory. Won’t work out in practice, because this produces code that will run, and it’s AI, so it must be good, right?

When you found code on the internet, it was presented in a context that let you make better judgement (e.g. on Stack Overflow this regular expression would have had a score of roughly −∞ and multiple highly-voted comments saying “do not use this, it’s catastrophically bad”), and where you have to put in more effort to plug it in and shuffle things around a bit as well. With Copilot, you get given ready-to-go code without any sanity checking at all.

See even how, a few seconds later in the video, the author does test it out—but not thoroughly enough.

Re: The Good and the Limitations of Github Copilot

#15

Post by tlrobinson » Mon, Jul 05, 2021, 5:15 AM UTC

nicetryguy wrote: post #3

Is === really necessary for Javascript? Type checking seems like overkill?

It’s a well known rule in JavaScript to default to “===“ for everything except checking for null or undefined, in which case “== null” is acceptable.

https://eslint.org/docs/rules/eqeqeq#smart

Even better is to just use TypeScript.

Re: The Good and the Limitations of Github Copilot

#16

Post by minimaxir » Mon, Jul 05, 2021, 5:45 AM UTC

A fun issue I keep hitting with Github Copilot in Python is that it's a coin flip whether it will give me a Python 3-style print statement or a Python 2.7 style print statement.

Re: The Good and the Limitations of Github Copilot

#17

Post by wildpeaks » Mon, Jul 05, 2021, 5:46 AM UTC

One problem is the lack of context: unlike Stackoverflow, you don't get additional infos from other users who tried the same code.

The only feedback Copilot receives is whether you keep it or not, you can't tell it a few days later that it wasn't a good fit after all (whereas you can comment on a Stackoverflow answer).

In its current form, it amplifies bias whereas code needs accuracy.

Re: The Good and the Limitations of Github Copilot

#18

Post by na85 » Mon, Jul 05, 2021, 5:48 AM UTC

chovybizzass wrote: post #6

If they are using my code anywhere I expect to be compensated.

The new model pioneered by AirBNB and Uber is to break laws under the guise of "growth hacking" and then change the laws later once you're entrenched.

You won't be compensated. Corporations write the laws.

Re: The Good and the Limitations of Github Copilot

#19

Post by alephnan » Mon, Jul 05, 2021, 5:54 AM UTC

wildpeaks wrote: post #17

One problem is the lack of context: unlike Stackoverflow, you don't get additional infos from other users who tried the same code. The only feedback Copilot receives is whether you keep it or not, you can't tell it a few days later that it wasn't a good fit after all (whereas you can comment on a Stackoverflow answer). In its current form, it amplifies bias whereas code needs accuracy.

That seems like an additional feature. Maybe append some comments to caveat the code snippet

Re: The Good and the Limitations of Github Copilot

#20

Post by hrithwikbha » Mon, Jul 05, 2021, 5:54 AM UTC

chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Hey Chris I am the author.

When i made the video I didn't really notice the code part of regex, since I am really new to regex but in the conclusion part of my video I did mention that most of the code is not efficient

Your comment was a great learning . Thank you