Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Good and the Limitations of Github Copilot

blog.hrithwik.me

Post reply on HN
Print view View original

121–130 of 144 posts

Re: The Good and the Limitations of Github Copilot

#121

Post by victor106 »

lilyball wrote: post #25
hrithwikbha wrote: post #20

Earlier quoted context omitted.

Hey Chris I am the author. When i made the video I didn't really notice the code part of regex, since I am really new to regex but in the conclusion part of my video I did mention that most of the code is not efficient Your comment was a great learning . Thank you

This is exactly the problem. The regex issue isn’t that it’s not efficient, it’s that it’s wrong. Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous.

Very well put. Thank You.

I know copilot is in alpha and will improve 100x but you will still need someone qualified to double check

Re: The Good and the Limitations of Github Copilot

#122

Post by corobo »

scoopertrooper wrote: post #112
corobo wrote: post #104

Earlier quoted context omitted.

Could you explain why or do you just want to feel superior? Both good choices honestly but I'd love to know your actual thoughts If I'm wrong tell me why and I'll happily reevaluate my opinion, promise :)

Are you aware co-pilot is still in preview? It's a bit harsh to make sweeping statements along the lines of 'it's just a fancy markov bot' based on a few well-publicised glitches in a technical preview. I assume you have built something surpassing the scope and ambition of co-pilot before, not just some armchair tech lead throwing shade.

Fair point on the sweeping statement, I'll rein it in

Thinking about it the main factor is an emotional one. I'm disappointed. Butthurt if you will. It sounded great but it tripped over so far away from the finish line that I've turned against it. I'll excuse myself from any further copilot threads

Of course I can never meet the requirements of your post wanting something more impressive than Copilot. Copilot itself falls far short of that. Nothing I give you will be enough as there'll be flaws you will attack to make your point. Bit of a time sink that, lets just assume you're right :)

No, I've never successfully built anything as ambitious as, and definitely nothing surpassing, copilot. Have I tried? Absolutely. Have I failed? So far yes.

Of course by that logic though I still win this conversation if you've not succeeded in making anything more ambitious than my failed projects, is that correct? :P

For the sake of not wanting to come off as blagging (also I want the holes poked in this one tbf) my most ambitious project I've not figured out how to make work yet is a new (afaik) type of business model: cohan.me/profit-share

Re: The Good and the Limitations of Github Copilot

#123

Post by ok2938 »

I'm all for moving forward, but Copilot just seems to be a bad idea, amplified.

What I want is careful, thoughtful, knowledgeable people, who have learned and honed their skills over years in various areas and can come up with creative, maintainable solutions to complex problems. This is not something you autocomplete. If it would be, we could autocomplete 80% of all jobs tomorrow.

I don't want code monkeys on steroids. But maybe I'm to far off Silicon Valley.

Re: The Good and the Limitations of Github Copilot

#124

Post by corobo »

make3 wrote: post #118
corobo wrote: post #104

Earlier quoted context omitted.

Could you explain why or do you just want to feel superior? Both good choices honestly but I'd love to know your actual thoughts If I'm wrong tell me why and I'll happily reevaluate my opinion, promise :)

Well, my understanding is that your point is that it's not useful and only regurgitates code it already saw. That's super false, it's useful tool that generates code in a very context sensitive manner. Not always perfect, and in alpha, but still very useful. The full copying only results from people actively probing the model to output copies of the code, and they made it work for very famous code that's been copied…

> that generates code in a very context sensitive manner

That's why I compared it to a markov chain aye

In any case I've come to the conclusion my hater attitude is fueled by disappointment, so maybe it was the worst take from this whole thing. I'll avoid future copilot threads

Re: The Good and the Limitations of Github Copilot

#125

Post by shadowgovt »

cf499 wrote: post #120
xoudini wrote: post #115

Earlier quoted context omitted.

Well, to be fair, the `len` operation on lists in Python is a constant time operation. What makes the example particularly bad is using a cast on the result of a floating-point division, rather than just using Python 3 integer division (i.e. the `//` operator). Copilot was clearly just spitting out Python 2 code here.

I think the difference is that redundant re-calculation is a code smell in any language, at all times, whereas the float/int division issue requires knowledge of Python 2/3 syntax quirks.

It's context-specific whether recalculation is a code smell. I've definitely gotten feedback from senior developers to just do len(x) repeatedly instead of "littering" (their words) the code with `xCount=len(x)` assignments (because they knew len was constant-time).

Re: The Good and the Limitations of Github Copilot

#126

Post by rwallace »

alkonaut wrote: post #39
chrismorgan wrote: post #8

> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 , here transcribed: /^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/ For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.…

This has been discussed at length several times before and the answer to why you use a regex like that to validate emails is because you aren't trying to validate against a standard, but against a subset of email formats. You want a "normal" simple email. No "+" domains etc. Because it doesn't matter if you annoy the 0.01% of your users that would be negatively affected by that, it's better to have their simple/canon…

The point being missed here is that no algorithm can tell you whether a string is a valid email address, because that's not a property of the string in the first place, it's a property of the world.

Having a fairly low entropy Gmail address, I get an intermittent drizzle of messages of the form of 'thank you for signing up to Acme!' whose content is such as to make it clear that an actual Acme customer typo'd their email address, and Acme thought you could validate it by checking the form of the string.

The only way to validate an email address is to send email to that address, asking the person behind it, are you the one who just signed up for Acme. And once you are doing that, there is no point checking the string for anything other than containing an @.

Re: The Good and the Limitations of Github Copilot

#127

Post by cf499 »

shadowgovt wrote: post #125
cf499 wrote: post #120

Earlier quoted context omitted.

I think the difference is that redundant re-calculation is a code smell in any language, at all times, whereas the float/int division issue requires knowledge of Python 2/3 syntax quirks.

It's context-specific whether recalculation is a code smell. I've definitely gotten feedback from senior developers to just do len(x) repeatedly instead of "littering" (their words) the code with `xCount=len(x)` assignments (because they knew len was constant-time).

Sure, it makes sense if you find something like `len_x` or `x_count` significantly less readable than `len(x)` and if you don't have to worry about resources. Can't say I've been there though.

I did a quick test of this `reverse` function (which probably shouldn't exist in the first place) and, unsurprisingly, it became ~30% faster when `len(arr)` was only called once.

Re: The Good and the Limitations of Github Copilot

#128

Post by shadowgovt »

cf499 wrote: post #127
shadowgovt wrote: post #125

Earlier quoted context omitted.

It's context-specific whether recalculation is a code smell. I've definitely gotten feedback from senior developers to just do len(x) repeatedly instead of "littering" (their words) the code with `xCount=len(x)` assignments (because they knew len was constant-time).

Sure, it makes sense if you find something like `len_x` or `x_count` significantly less readable than `len(x)` and if you don't have to worry about resources. Can't say I've been there though. I did a quick test of this `reverse` function (which probably shouldn't exist in the first place) and, unsurprisingly, it became ~30% faster when `len(arr)` was only called once.

When I'm that resource constrained, I don't use Python. The changes necessary to make regular Python code performant defeat the goal of making it readable.

There's a reason NumPy's innards aren't Python code.

Re: The Good and the Limitations of Github Copilot

#129

Post by alkonaut »

rwallace wrote: post #126
alkonaut wrote: post #39

Earlier quoted context omitted.

This has been discussed at length several times before and the answer to why you use a regex like that to validate emails is because you aren't trying to validate against a standard, but against a subset of email formats. You want a "normal" simple email. No "+" domains etc. Because it doesn't matter if you annoy the 0.01% of your users that would be negatively affected by that, it's better to have their simple/canon…

The point being missed here is that no algorithm can tell you whether a string is a valid email address, because that's not a property of the string in the first place, it's a property of the world. Having a fairly low entropy Gmail address, I get an intermittent drizzle of messages of the form of 'thank you for signing up to Acme!' whose content is such as to make it clear that an actual Acme customer typo'd their e…

> The point being missed here is that no algorithm can tell you whether a string is a valid email address, because that's not a property of the string in the first place, it's a property of the world

"Valid" can mean at least 3 different things:

a) Conformant to a spec

b) Can actually receive email

c) Looks like a nice, simple "canonical" standard email address.

If you validate to the RFC (a) you still might fail b) and c). (The value of c is debated at length in a separate subthread but let'sjust say that there are more or less shady reasons why this is often a business goal).

Since you'll probably validate b) anyway - the validation of either b) or c) is a convenience, because validating b) isn't instant. So you validate to prevent errors and frustration. The question is merely: do I as a business want to have an address with quotes, spaces and backslashes in it, in my database just because it's possible according to the specification?

> And once you are doing that, there is no point checking the string for anything other than containing an @.

I think there is a legitmate case for a service to simply think "I'd rather lose the business of 1 customer out of a million than worry about backslashes in email addresses". It's not user friendly, and it's not "correct", but it's one of those "good enough" scenaroios.

Re: The Good and the Limitations of Github Copilot

#130

Post by make3 »

corobo wrote: post #124
make3 wrote: post #118

Earlier quoted context omitted.

Well, my understanding is that your point is that it's not useful and only regurgitates code it already saw. That's super false, it's useful tool that generates code in a very context sensitive manner. Not always perfect, and in alpha, but still very useful. The full copying only results from people actively probing the model to output copies of the code, and they made it work for very famous code that's been copied…

> that generates code in a very context sensitive manner That's why I compared it to a markov chain aye In any case I've come to the conclusion my hater attitude is fueled by disappointment, so maybe it was the worst take from this whole thing. I'll avoid future copilot threads

markov chains model n grams.. this is really much better
Post reply on HN