Live data from Hacker News

The Good and the Limitations of Github Copilot

blog.hrithwik.me

21–30 of 144 posts

Re: The Good and the Limitations of Github Copilot

#23
post #10

Earlier quoted context omitted.

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

Nice theory. Won’t work out in practice, because this produces code that will run , and it’s AI, so it must be good, right? When you found code on the internet, it was presented in a context that let you make better judgement (e.g. on Stack Overflow this regular expression would have had a score of roughly −∞ and multiple highly-voted comments saying “do not use this, it’s catastrophically bad”), and where you have t…

I think Copilot should report the matching source URL to allow the user to visit the page and see the context and license. This move would also placate some copyright questions because it would be like searching StackOverflow or Github for inspiration.

The problem of content attribution (exact and fuzzy match) has been studied before under the task of plagiarism detection for student essays. Funny thing is that a plagiarism detection Copilot would also disclose past cases of copyright violation and cause attribution disputes because code sitting unchecked in various repos would suddenly become visible.

Re: The Good and the Limitations of Github Copilot

#25

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Hey Chris I am the author. When i made the video I didn't really notice the code part of regex, since I am really new to regex but in the conclusion part of my video I did mention that most of the code is not efficient Your comment was a great learning . Thank you

This is exactly the problem. The regex issue isn’t that it’s not efficient, it’s that it’s wrong. Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous.

Re: The Good and the Limitations of Github Copilot

#26
post #10

Earlier quoted context omitted.

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

Nice theory. Won’t work out in practice, because this produces code that will run , and it’s AI, so it must be good, right? When you found code on the internet, it was presented in a context that let you make better judgement (e.g. on Stack Overflow this regular expression would have had a score of roughly −∞ and multiple highly-voted comments saying “do not use this, it’s catastrophically bad”), and where you have t…

Decently puts my feelings towards this whole thing

Aside from the beaten horse concerns like licensing... I worry about the training we're giving ourselves and future generations

The upfront presentation of 'suggestions' skews the perception, a fair bit of 'no warranty guaranteed' comes from having to go dig it up

Re: The Good and the Limitations of Github Copilot

#27
post #10

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

Copilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.

Search engines give you a link where you can (usually) see the code in context, who wrote it, when, license, etc. And often more, like who is using it where, how often it's updated, contact info, test suites, and so on.

Re: The Good and the Limitations of Github Copilot

#28

Earlier quoted context omitted.

Hey Chris I am the author. When i made the video I didn't really notice the code part of regex, since I am really new to regex but in the conclusion part of my video I did mention that most of the code is not efficient Your comment was a great learning . Thank you

This is exactly the problem. The regex issue isn’t that it’s not efficient, it’s that it’s wrong. Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous.

> Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous.

I would like this message to be amplified as much as possible. Never write code you do not understand. I am excited about copilot, but also wary of the programming culture these tools will bring in. Businesses, especially body-shopping companies will want to deliver as much using tools in this category and end up shipping code with disastrous edge cases.

Re: The Good and the Limitations of Github Copilot

#30
post #23

Earlier quoted context omitted.

Nice theory. Won’t work out in practice, because this produces code that will run , and it’s AI, so it must be good, right? When you found code on the internet, it was presented in a context that let you make better judgement (e.g. on Stack Overflow this regular expression would have had a score of roughly −∞ and multiple highly-voted comments saying “do not use this, it’s catastrophically bad”), and where you have t…

I think Copilot should report the matching source URL to allow the user to visit the page and see the context and license. This move would also placate some copyright questions because it would be like searching StackOverflow or Github for inspiration. The problem of content attribution (exact and fuzzy match) has been studied before under the task of plagiarism detection for student essays. Funny thing is that a pla…

> I think Copilot should report the matching source URL

That's the problem. The output of a GAN like Copilot usually can't be traced directly back to a single input.

Post reply on HN