Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

40% of the code GitHub Copilot users check-in is AI generated and unmodified

microsoft.com

Post reply on HN
Print view View original

71–80 of 129 posts

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#71

Post by steve1977 »

paxys wrote: post #38
monero-xmr wrote: post #28

There is a lot of annoyance in the boilerplate of a program - "set up a scaffold for this, template that, convert data from this format to that format". The AI tools I have experimented with are good at this and for sure this is useful. But no one can read my mind! If I have to get increasingly specific in what I am telling the AI, and have to formulate my writing style in a way that keeps things very precise, I am e…

That was my exact thought. I don't use copilot but I'd wager that at least 40% of the code I check-in is autogenerated as well. A fresh modern JS project is 15+ files full of boilerplate around a single "hello, world" statement. If there was a metric for how much net new application business logic was written by copilot, that would be far more interesting to study.

It would also be interesting to see how it does in less “polluted” programming environments than web frontend.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#72

Post by steve1977 »

symlinkk wrote: post #41

All of you should stop posting your knowledge of programming online (Hacker News, Stack Overflow, Reddit, blogs, etc) unless you want an AI to absorb it and take your job from you.

I guess they should primarily stop using GitHub as a source repository.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#73

Post by zarzavat »

sph wrote: post #69

Aside: am I the only one here that is still writing code by hand and not using AI to autogenerate it? Because it seems over the past couple years on HN everybody is making great use of Copilot and constantly talking about it, while honestly scaffolding boilerplate is the least of my problems, while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my t…

Copilot is way more than scaffolding boilerplate, it catches bugs as you are writing them.

For example you expect: x[i]

But copilot suggests: x[i+1]

…and you think, and you think, and then one of two things happens. Either you satisfy yourself that the answer is indeed x[i], or you accept x[i+1] and sheepishly thank Copilot for catching a bug today.

You could have copilot on in the background and never accept a suggestion, and it’s still a productivity enhancement.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#74

Post by [deleted] »

xtreme wrote: post #46
Swizec wrote: post #36

Earlier quoted context omitted.

> It is possible that someday it reads in an entire codebase (say, millions of lines of code), and I can describe a new API I need and it implements the entire thing. This would be quite useful. You can get this today by switching to being a product manager/owner! Dream up an idea, describe it to a bunch of fuzzy intelligences, go through a few cycles of refinement, and voila: A software.

Only problem is that they cost $10,000/mo instead of $10/mo.

[deleted]

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#75

Post by jaredcwhite »

sph wrote: post #69

Aside: am I the only one here that is still writing code by hand and not using AI to autogenerate it? Because it seems over the past couple years on HN everybody is making great use of Copilot and constantly talking about it, while honestly scaffolding boilerplate is the least of my problems, while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my t…

Indeed you are not. I've never used any generative AI tools when coding or otherwise, and continue to refuse to at this point in time.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#76

Post by nathansherburn »

zarzavat wrote: post #73
sph wrote: post #69

Aside: am I the only one here that is still writing code by hand and not using AI to autogenerate it? Because it seems over the past couple years on HN everybody is making great use of Copilot and constantly talking about it, while honestly scaffolding boilerplate is the least of my problems, while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my t…

Copilot is way more than scaffolding boilerplate, it catches bugs as you are writing them. For example you expect: x[i] But copilot suggests: x[i+1] …and you think, and you think, and then one of two things happens. Either you satisfy yourself that the answer is indeed x[i], or you accept x[i+1] and sheepishly thank Copilot for catching a bug today. You could have copilot on in the background and never accept a sugge…

I found during my preview months ago that it would often write bugs that took me longer to correct than if I had written it myself. Has it gotten better?

Also in the situation where it suggests x[i+1] and it's wrong, doesn't that annoy you?

In both cases I'm sure this is just temporary while LLMs get better. But my experience was they were _just_ a little too annoying last time I used them.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#77

Post by tasuki »

sph wrote: post #69

Aside: am I the only one here that is still writing code by hand and not using AI to autogenerate it? Because it seems over the past couple years on HN everybody is making great use of Copilot and constantly talking about it, while honestly scaffolding boilerplate is the least of my problems, while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my t…

> thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my time

How can you be sure of that?

I know scientists who use ChatGPT to rewrite their text to be clearer. They've also used it to pose novel interesting questions about a situation.

On my side, I've found ChatGPT very helpful to explore/compare which library to use to solve my problem and show examples how that library is used.

It's also been very helpful lining up Haskell types and translating GHC error messages into English. Sure, those are non-creative tasks, but it's surprisingly good at them.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#78

Post by ramblerman »

sph wrote: post #69

Aside: am I the only one here that is still writing code by hand and not using AI to autogenerate it? Because it seems over the past couple years on HN everybody is making great use of Copilot and constantly talking about it, while honestly scaffolding boilerplate is the least of my problems, while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my t…

> while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my time.

So don't use it there :) But I'm curious what kind of coding you do that you never have to do any scaffolding, refactoring, or come across any element of repetitiveness.

That is where it shines imo. I find during those boring tasks it quite often just gets my intent, and I can say "yes actually that's exactly what I wanted".

Which allows me to get back to the business of "thinking, designing and abstracting novel ideas" quicker.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#79

Post by worrycue »

tasuki wrote: post #77
sph wrote: post #69

Aside: am I the only one here that is still writing code by hand and not using AI to autogenerate it? Because it seems over the past couple years on HN everybody is making great use of Copilot and constantly talking about it, while honestly scaffolding boilerplate is the least of my problems, while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my t…

> thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my time How can you be sure of that? I know scientists who use ChatGPT to rewrite their text to be clearer. They've also used it to pose novel interesting questions about a situation. On my side, I've found ChatGPT very helpful to explore/compare which library to use to solve my problem and show examp…

Can you trust it though?

It can’t really reason - it’s just very good at guessing the right answer if there are a lot of examples.

It also lies somewhat frequently.

If we have to double check its output, it kind of defeats the purpose - at least part of it - as the “cognitive burden” is still on us.

P.S. Not sure what you mean by “lining up” Haskell types … but don’t LSPs already give us clear information about the inferred types and do so with nigh zero chance of errors.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#80

Post by sph »

ramblerman wrote: post #78
sph wrote: post #69

Aside: am I the only one here that is still writing code by hand and not using AI to autogenerate it? Because it seems over the past couple years on HN everybody is making great use of Copilot and constantly talking about it, while honestly scaffolding boilerplate is the least of my problems, while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my t…

> while thinking, designing and abstracting novel ideas into code is the hard part where AI would fall short and just waste my time. So don't use it there :) But I'm curious what kind of coding you do that you never have to do any scaffolding, refactoring, or come across any element of repetitiveness. That is where it shines imo. I find during those boring tasks it quite often just gets my intent, and I can say "yes…

AI is not as foolproof as many want us to believe. In my mind it's like giving a less experienced and error-prone engineer the code I'm working on to refactor.

Perhaps it's because I've always been a lone wolf engineer, but I'd rather do it myself than giving quite an important task to someone less experienced than me, whether I'm writing something new (where I need to be at my best) or refactoring (where I still need to be sharp to understand why it was done a certain way to know how to improve)

AI would help with extracting code into a function, or writing yet another algorithm implementation, or wiring bog standard libraries (say, cURL), but that's like foreplay and lower effort work I enjoy doing as a break from the hard stuff.

And to clarify my comment about AI being an engineer with less experience than me, I'd rather be paired with an actual junior engineer, so at least I can mentor them and help them grow, than just having a machine do something that in some cases might be utterly inane, so I still have to waste time double checking its work, but totally powerless to teach and improve it.

Post reply on HN