Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

40% of the code GitHub Copilot users check-in is AI generated and unmodified

microsoft.com

Post reply on HN
Print view View original

31–40 of 129 posts

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#31

Post by aaomidi »

paro_nej wrote: post #29
dandellion wrote: post #21

Earlier quoted context omitted.

In my case about 50% of the test code is just boilerplate, so I usually type the test name and generate the rest. Most of the time I have to rewrite the actual test logic (although very trivial tests are sometimes correct), but I probably keep more than 40% if we're talking tests specifically.

> In my case about 50% of the test code is just boilerplate Why not replace it with a function call?

Some languages are just noisy.

Go, Java I’ve experienced are noisier than others.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#32

Post by cortesoft »

luxurytent wrote: post #5

I'm not surprised. A lot of the code I'll write in a day is trivial, calling helpers, writing test scaffolding, etc. I'm still building cool things the other 60% of time, but this job as we know it comes with a lot of repetition. (esp. if you write unit tests ;-))

Yeah, to me this doesn’t feel that different than using a library from another developer, just slightly more customized. Most code that is run is not written by the final developer, but by people further up the chain.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#33

Post by aaomidi »

r3dneck wrote: post #26

IMHO, if you really "tell a CEO I can give him 56% more developers overnight" , he will sue you. Every programmer tried it out can tell that his affirmation could be right only for trivial part of a program, for non trivial programming part, it is more like a board game, where you try to induce the "AI" to produce the right answer modifying the "prompt" again, again, again, again, ( Did I tell you "again" ?) and agai…

I mean this is a prime example why lines of code are a terrible metric.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#34

Post by travisjungroth »

paro_nej wrote: post #29
dandellion wrote: post #21

Earlier quoted context omitted.

In my case about 50% of the test code is just boilerplate, so I usually type the test name and generate the rest. Most of the time I have to rewrite the actual test logic (although very trivial tests are sometimes correct), but I probably keep more than 40% if we're talking tests specifically.

> In my case about 50% of the test code is just boilerplate Why not replace it with a function call?

A common occurrence is something like a pool of 10 actions where a bunch of tests each do 3 to 7 of them. This is very hard to abstract with a function call.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#36

Post by Swizec »

monero-xmr wrote: post #28

There is a lot of annoyance in the boilerplate of a program - "set up a scaffold for this, template that, convert data from this format to that format". The AI tools I have experimented with are good at this and for sure this is useful. But no one can read my mind! If I have to get increasingly specific in what I am telling the AI, and have to formulate my writing style in a way that keeps things very precise, I am e…

> It is possible that someday it reads in an entire codebase (say, millions of lines of code), and I can describe a new API I need and it implements the entire thing. This would be quite useful.

You can get this today by switching to being a product manager/owner! Dream up an idea, describe it to a bunch of fuzzy intelligences, go through a few cycles of refinement, and voila: A software.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#38

Post by paxys »

monero-xmr wrote: post #28

There is a lot of annoyance in the boilerplate of a program - "set up a scaffold for this, template that, convert data from this format to that format". The AI tools I have experimented with are good at this and for sure this is useful. But no one can read my mind! If I have to get increasingly specific in what I am telling the AI, and have to formulate my writing style in a way that keeps things very precise, I am e…

That was my exact thought. I don't use copilot but I'd wager that at least 40% of the code I check-in is autogenerated as well. A fresh modern JS project is 15+ files full of boilerplate around a single "hello, world" statement.

If there was a metric for how much net new application business logic was written by copilot, that would be far more interesting to study.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#39

Post by rossjudson »

Swizec wrote: post #36
monero-xmr wrote: post #28

There is a lot of annoyance in the boilerplate of a program - "set up a scaffold for this, template that, convert data from this format to that format". The AI tools I have experimented with are good at this and for sure this is useful. But no one can read my mind! If I have to get increasingly specific in what I am telling the AI, and have to formulate my writing style in a way that keeps things very precise, I am e…

> It is possible that someday it reads in an entire codebase (say, millions of lines of code), and I can describe a new API I need and it implements the entire thing. This would be quite useful. You can get this today by switching to being a product manager/owner! Dream up an idea, describe it to a bunch of fuzzy intelligences, go through a few cycles of refinement, and voila: A software.

And now on to the other 99% of the problem. ;)

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#40

Post by thatswrong0 »

paxys wrote: post #38
monero-xmr wrote: post #28

There is a lot of annoyance in the boilerplate of a program - "set up a scaffold for this, template that, convert data from this format to that format". The AI tools I have experimented with are good at this and for sure this is useful. But no one can read my mind! If I have to get increasingly specific in what I am telling the AI, and have to formulate my writing style in a way that keeps things very precise, I am e…

That was my exact thought. I don't use copilot but I'd wager that at least 40% of the code I check-in is autogenerated as well. A fresh modern JS project is 15+ files full of boilerplate around a single "hello, world" statement. If there was a metric for how much net new application business logic was written by copilot, that would be far more interesting to study.

I just started using Copilot and so far, it really does seem to save me time with rote stuff like imports, function definitions, and where I would otherwise be copying / pasting code I just wrote (say, while writing a bunch of tests). Even when it gets those slightly wrong, it still saves me time or at least makes it feel that way.

So overall I'm honestly pleasantly surprised with it. It feels similar to when I got a car with adaptive cruise control - suddenly I'm not thinking about maintaining the correct speed and following distance on the highway, which makes driving for a while require far less mental effort.

I know I probably could have accomplished something similar by setting up snippets or whatever, but I installed it and it immediately started doing what I wanted without having to ask.

Nifty.

Post reply on HN