Live data from Hacker News

The Good and the Limitations of Github Copilot

blog.hrithwik.me

51–60 of 144 posts

Re: The Good and the Limitations of Github Copilot

#51
post #33

I wonder how much worse or better simply automating a stackexchange search and pasting the results is...

Searching SO and converting the highest rated answer's code that doesn't syntax error to a Python module is about 150 LoC[0]. [0]: https://github.com/drathier/stack-overflow-import

Horrifying and beautiful at the same time!

Re: The Good and the Limitations of Github Copilot

#52

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

> It generates a nastily complex regular expression that is hopelessly wrong.

> [...]

> The author makes no comment on how hideously bad it is[...]

I mean, it's coming up with a solution that's about as good as the average programmer who's going to validate E-Mail with regexes would, so as a crowd-sourced machine learning solution it's not too bad if you think about it.

In other words, Having a co-pilot doesn't mean you're guaranteed to get Chuck Yeager.

Re: The Good and the Limitations of Github Copilot

#53
post #52

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

> It generates a nastily complex regular expression that is hopelessly wrong. > [...] > The author makes no comment on how hideously bad it is[...] I mean, it's coming up with a solution that's about as good as the average programmer who's going to validate E-Mail with regexes would, so as a crowd-sourced machine learning solution it's not too bad if you think about it. In other words, Having a co-pilot doesn't mean…

Here’s the difference imo, an average programmer with some experience would seek out well tested and used library to help them do something like this, not use sausage meet spat out of a cannon to validate email addresses.

Re: The Good and the Limitations of Github Copilot

#54
post #23

Earlier quoted context omitted.

I think Copilot should report the matching source URL to allow the user to visit the page and see the context and license. This move would also placate some copyright questions because it would be like searching StackOverflow or Github for inspiration. The problem of content attribution (exact and fuzzy match) has been studied before under the task of plagiarism detection for student essays. Funny thing is that a pla…

> I think Copilot should report the matching source URL That's the problem. The output of a GAN like Copilot usually can't be traced directly back to a single input.

If you can't trace the source then it's transformative use. If it matches training data then it needs to report the source like a search engine and place all responsibility on the user.

And fuzzy code matching could be easily implemented by using the a model similar to CLIP (contrastive) to embed code snippets.

Re: The Good and the Limitations of Github Copilot

#55
post #49

Earlier quoted context omitted.

This has been discussed at length several times before and the answer to why you use a regex like that to validate emails is because you aren't trying to validate against a standard, but against a subset of email formats. You want a "normal" simple email. No "+" domains etc. Because it doesn't matter if you annoy the 0.01% of your users that would be negatively affected by that, it's better to have their simple/canon…

How is it better to have "simple" emails?

It depends on the use case, but for some business use cases you e.g prefer access to the end users default/canonical inbox and not a specific one the user can use to filter etc. It’s also much less likely to cause downstream problems like distribution issues, be rejection by spam filters and so on. Annoying a tiny fraction of users or losing their business just isn’t a big enough issue to matter.

Re: The Good and the Limitations of Github Copilot

#57
post #7

If they are using my code anywhere I expect to be compensated.

If you read the license you put on your project (and/or the GitHub Terms of Service), you wouldn't expect that.

Huge leap of logic to assume what license another user publishes their code under. A charitable reading here of chovybizzass's comment leads me to the understanding that they are publishing their code under a license that requires compensation if used, this code can be published to GitHub as well. Would it be stupid? Probably. But even more stupid would be to produce MIT-licensed code and then complain about not being compensated.

Re: The Good and the Limitations of Github Copilot

#59

A fun issue I keep hitting with Github Copilot in Python is that it's a coin flip whether it will give me a Python 3-style print statement or a Python 2.7 style print statement.

Most instances of that should be fixable with some post-processing in Copilot. They could maybe even run py2to3 on it?

Re: The Good and the Limitations of Github Copilot

#60
post #49

Earlier quoted context omitted.

This has been discussed at length several times before and the answer to why you use a regex like that to validate emails is because you aren't trying to validate against a standard, but against a subset of email formats. You want a "normal" simple email. No "+" domains etc. Because it doesn't matter if you annoy the 0.01% of your users that would be negatively affected by that, it's better to have their simple/canon…

How is it better to have "simple" emails?

This is usually done to catch what they think has a high chance of being a user error, even if otherwise valid.
Post reply on HN