Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did GitHub Copilot increase my productivity?

trace.yshui.dev

Post reply on HN
Print view View original

311–320 of 326 posts

Re: Did GitHub Copilot increase my productivity?

#311

Post by munksbeer »

zarathustreal wrote: post #254
klauserc wrote: post #148

I find GitHub Copilot close to useless for production code. The worst, most obscure bugs I've had to debug in the last year were all in Copilot-written code. It _looks_ plausible, but it makes extremely subtle mistakes. Occasionally, you have repetitive sections of code where it can copy&adapt lines from the context, but that's about it. It's a different story for test code. Test code is often formulaic and "standard…

If you’re not using a language that can properly support algebraic structures and randomized property-based testing you’re essentially getting no guarantees about your code from tests. You wrote the code, you wrote the tests, they’re equally likely to be incorrect.

I find this to be a bit of a meaningless point. What are you actually trying to say?

Re: Did GitHub Copilot increase my productivity?

#312

Post by zarathustreal »

munksbeer wrote: post #311
zarathustreal wrote: post #254

Earlier quoted context omitted.

If you’re not using a language that can properly support algebraic structures and randomized property-based testing you’re essentially getting no guarantees about your code from tests. You wrote the code, you wrote the tests, they’re equally likely to be incorrect.

I find this to be a bit of a meaningless point. What are you actually trying to say?

My statement is clear and straightforward, I’m not sure how to put it any other way. LLM-generated tests don’t make sense as a concept because there are only roughly five properties you actually need to write tests for if you’re writing tests that actually provide any guarantees.

Re: Did GitHub Copilot increase my productivity?

#313

Post by SOF3 »

Not targeting anyone, but after clicking into the link, I (1) noticed it is mdbook, (2) looked at the author GitHub and noticed C and Rust, and (3) predicted that Copilot is gonna be unpredictable with random serious unnoticeable mistakes. Looks like I got it right.

Re: Did GitHub Copilot increase my productivity?

#314

Post by munksbeer »

zarathustreal wrote: post #312
munksbeer wrote: post #311

Earlier quoted context omitted.

I find this to be a bit of a meaningless point. What are you actually trying to say?

My statement is clear and straightforward, I’m not sure how to put it any other way. LLM-generated tests don’t make sense as a concept because there are only roughly five properties you actually need to write tests for if you’re writing tests that actually provide any guarantees.

Apologies, but I understand the English words you're typing but I'm still not sure of the intent you're trying to convey to everyone. You're conversing in a very rigid style which isn't sympathetic to how people typically interact.

I could just leave the discussion I guess, but in the interest of discourse, I don't find your statement meaningful because we're not all working in languages that I think you refer to. Our unit tests are absolutely not perfect and don't offer perfect guarantees, as we're fallible and will write fallible code.

And as such, I just don't understand what point you're trying to make by saying that LLM generated tests are no good because they can't offer perfect guarantees.

Re: Did GitHub Copilot increase my productivity?

#315

Post by collyw »

devjab wrote: post #164
klauserc wrote: post #148

I find GitHub Copilot close to useless for production code. The worst, most obscure bugs I've had to debug in the last year were all in Copilot-written code. It _looks_ plausible, but it makes extremely subtle mistakes. Occasionally, you have repetitive sections of code where it can copy&adapt lines from the context, but that's about it. It's a different story for test code. Test code is often formulaic and "standard…

I think automated tests are the one area that LLMs will truly improve productivity (and overall code quality). It’ll likely also lead to a lot of tests that actually tests nothing, but as a whole, it’ll hopefully be capable of both generating and updating tests if you give it some good inputs to do it on. Documentation is another area where I have high hopes. In the ideal world people update it as they change things.…

If the code isn't doing anything special, it spits out decent enough code (I am using the paid version of ChatGPT with the various customization).

As someone who spends 80% of his time in the backend, I find it great for JavaScript whereas it's not so good for Django which I know pretty well.It can still be useful though and is often faster than looking up docs for specific things.

Re: Did GitHub Copilot increase my productivity?

#316

Post by collyw »

munksbeer wrote: post #311
zarathustreal wrote: post #254

Earlier quoted context omitted.

If you’re not using a language that can properly support algebraic structures and randomized property-based testing you’re essentially getting no guarantees about your code from tests. You wrote the code, you wrote the tests, they’re equally likely to be incorrect.

I find this to be a bit of a meaningless point. What are you actually trying to say?

Humble-brag about how he uses "a language that can properly support algebraic structures and randomized property-based testing" whatever the hell that is.

Personally I use python and solve real world problems.

Re: Did GitHub Copilot increase my productivity?

#317

Post by collyw »

999900000999 wrote: post #275

Chat GPT and friends are best when you tell it to write code from scratch, even then it's only good for straight forward tasks. Give it existing code to work with, particularly a larger code base and things get weird. Still, I love it for hacking out small prototypes. If you don't really care about performance or best practice, it's ok.

ChatGPT4 It won't accept moderately large files that I have given it. It was actually just an HTML file, I wanted to ask about css in it. How have you managed to give it a "larger codebase"?

Re: Did GitHub Copilot increase my productivity?

#318

Post by TheRealDunkirk »

It increased mine yesterday, by quite a bit. I was writing some JSON-to-array-of-arrays conversion code, and when I started to type it out -- dreading the inevitable off-by-one hunt -- CoPilot spat out the code, and it was correct! It helped that I've done this many times, and knew it was correct, but it saved me needing to think about it, or track down any bugs.

This conversion code was in service to using Plotly (the JS graphing library) for the first time. I was trying to get the axes of my surface plot to be labelled with the correct values (instead of just their index), and spent probably a half hour looking at the demos and the docs, and not finding what I needed. I asked CoPilot in the VSCode chat window about it, and it gave me exactly what I hadn't been able to find. As soon as I saw it, I understood the natural extension of the API call I was seeing in the demo, but I still hadn't found a second data point to even start extrapolating, if you take my meaning.

Anyway, I'm paying the $10/mo out of my own pocket, and yesterday -- at least to me -- it paid for itself for the month in just those two examples. I find it delightfully surprising. I just need to find a way to make it stop suggesting comments. I can type my own comments, thank you very much.

Re: Did GitHub Copilot increase my productivity?

#319

Post by doubloon »

TillE wrote: post #237
doubloon wrote: post #220

The answer is yes for most people. You can type in pseudocode for simple things and out pops something that basically works for a significant number of languages. Its absolutely terrible for really complex problems but most code is not solving complex problems.

> but most code is not solving complex problems I see this claim a lot, and I have no idea whether or not it's true. I just struggle to imagine that there are millions of people employed, well compensated for doing pretty easy jobs. In my professional work and my various unrelated personal projects and open source contributions, if you want to do anything useful or interesting, you get into the weeds very very quickl…

Its opaque because its operations. Where vba and excel come in

Re: Did GitHub Copilot increase my productivity?

#320

Post by 999900000999 »

collyw wrote: post #317
999900000999 wrote: post #275

Chat GPT and friends are best when you tell it to write code from scratch, even then it's only good for straight forward tasks. Give it existing code to work with, particularly a larger code base and things get weird. Still, I love it for hacking out small prototypes. If you don't really care about performance or best practice, it's ok.

ChatGPT4 It won't accept moderately large files that I have given it. It was actually just an HTML file, I wanted to ask about css in it. How have you managed to give it a "larger codebase"?

By just giving it code files one by one.

By larger I'm talking about maybe 5 or 6 files.

Or to be honest Chat GPT isn't good when your fixing code that already exists.

Post reply on HN