Live data from Hacker News

40% of the code GitHub Copilot users check-in is AI generated and unmodified

microsoft.com

41–50 of 129 posts

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#42
post #19

So far I’ve found copilot is like a weather forecast. The shorter the prediction the higher probability of it being correct. The longer, and more complex the prediction the less likely it’s all correct. But, with a close review you can spot the errors and correct it. I’ve found most useful to get ideas on the page fast. It’s also absolutely insane to be writing a parser, and have it suggest unit tests and code where…

the generalization is errors compounding over time

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#43

Read: 40% of the code Github Copilot users check-in is either boilerplate or has bugs in it.

true that. 40% of the code is AI generated doesn't mean any of it actually runs. Morely likely we're on the hype cycle of AI generated code right now.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#44
I am a PM and a novice programmer. Unless copilot improves drastically for me, I am going to cancel the subscription and keep the one for chatgpt pro for now.

I have been able to write fairly decent programs using chatgpt, am starting learn and build full stack apps etc. I am not able to use copilot. Maybe I do not know the tricks.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#45
post #44

I am a PM and a novice programmer. Unless copilot improves drastically for me, I am going to cancel the subscription and keep the one for chatgpt pro for now. I have been able to write fairly decent programs using chatgpt, am starting learn and build full stack apps etc. I am not able to use copilot. Maybe I do not know the tricks.

You have to get Copilot started, writing a comment is often enough to do so. It doesn’t write from prompts like ChatGPT of course.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#46
post #36

There is a lot of annoyance in the boilerplate of a program - "set up a scaffold for this, template that, convert data from this format to that format". The AI tools I have experimented with are good at this and for sure this is useful. But no one can read my mind! If I have to get increasingly specific in what I am telling the AI, and have to formulate my writing style in a way that keeps things very precise, I am e…

> It is possible that someday it reads in an entire codebase (say, millions of lines of code), and I can describe a new API I need and it implements the entire thing. This would be quite useful. You can get this today by switching to being a product manager/owner! Dream up an idea, describe it to a bunch of fuzzy intelligences, go through a few cycles of refinement, and voila: A software.

Only problem is that they cost $10,000/mo instead of $10/mo.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#47

I'm not surprised. A lot of the code I'll write in a day is trivial, calling helpers, writing test scaffolding, etc. I'm still building cool things the other 60% of time, but this job as we know it comes with a lot of repetition. (esp. if you write unit tests ;-))

The real question is, why we need to write "trivial" code? I think people have lost the plot there. We should be writing better compilers and libraries, instead of using AI that will write more code to deal with.

The obsession with code generation instead of creating proper abstractions is a scourge of today's programming. It's a regression in software engineering.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#48
post #46
post #36

Earlier quoted context omitted.

> It is possible that someday it reads in an entire codebase (say, millions of lines of code), and I can describe a new API I need and it implements the entire thing. This would be quite useful. You can get this today by switching to being a product manager/owner! Dream up an idea, describe it to a bunch of fuzzy intelligences, go through a few cycles of refinement, and voila: A software.

Only problem is that they cost $10,000/mo instead of $10/mo.

$/mo doesn't matter. $/function does.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#49
post #47

I'm not surprised. A lot of the code I'll write in a day is trivial, calling helpers, writing test scaffolding, etc. I'm still building cool things the other 60% of time, but this job as we know it comes with a lot of repetition. (esp. if you write unit tests ;-))

The real question is, why we need to write "trivial" code? I think people have lost the plot there. We should be writing better compilers and libraries, instead of using AI that will write more code to deal with. The obsession with code generation instead of creating proper abstractions is a scourge of today's programming. It's a regression in software engineering.

I agree to some extent. But there are redundancies that are inherently beneficial and make code more robust and readable, like separating interface from implementation (where some declarative elements have to be repeated), or explicit type annotations instead of full-program type inference. There are also strictly necessary redundancies, like when you pattern-match an ADT, you repeat its declared structure. For desirable or necessary redundancies like that, AI code completion can still save time.

Intelligent unit test creation (if it is indeed intelligent) is also a good use-case. Not to relieve the programmer from thinking, but to save typing and to point out potentially useful tests that you might not have thought of.

Re: 40% of the code GitHub Copilot users check-in is AI generated and unmodified

#50
post #47

I'm not surprised. A lot of the code I'll write in a day is trivial, calling helpers, writing test scaffolding, etc. I'm still building cool things the other 60% of time, but this job as we know it comes with a lot of repetition. (esp. if you write unit tests ;-))

The real question is, why we need to write "trivial" code? I think people have lost the plot there. We should be writing better compilers and libraries, instead of using AI that will write more code to deal with. The obsession with code generation instead of creating proper abstractions is a scourge of today's programming. It's a regression in software engineering.

The boilerplate makes it readable by humans.
Post reply on HN