Live data from Hacker News

I tested Copilot in an existing open-source codebase with unit tests [video]

youtube.com

11–15 of 15 posts

Re: I tested Copilot in an existing open-source codebase with unit tests [video]

#11
post #7

it would be interesting if copilot could generate the code based on your unit tests

Would that be more desirable than building your tests from your application?

Seems to me that once you can go either way then you just proved the other redundant.

Re: I tested Copilot in an existing open-source codebase with unit tests [video]

#12
post #7

it would be interesting if copilot could generate the code based on your unit tests

Would that be more desirable than building your tests from your application? Seems to me that once you can go either way then you just proved the other redundant.

The central thesis of this diatribe is that in order to have a 'complete' unit^ test, you need something at least as complicated as the original code:

https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste...

^ where the 'unit' is basically a function. i.e., to test a function fully, write a test as complicated as the function

Re: I tested Copilot in an existing open-source codebase with unit tests [video]

#13

Earlier quoted context omitted.

It's mediocre. Sometimes it's plain wrong, sometimes right but not exact and so on.

Was Copilot's value proposition that it gets better over time as more developers use it over a longer period?

But the value for who? Last time I checked the consensus was that using it in any project that is not for private use can get you in legal trouble. So for me it's completely useless, even if technically it could be improved with time (which may or may not happen).

Re: I tested Copilot in an existing open-source codebase with unit tests [video]

#14
post #8

He probably spent more time figuring out if the suggestions are usable than he would have needed to just implement those manually.

I guess if he wrote tests first it would have said. no method called x in util. TDD helps take a lot of thinking out of problems.

Re: I tested Copilot in an existing open-source codebase with unit tests [video]

#15

Earlier quoted context omitted.

Was Copilot's value proposition that it gets better over time as more developers use it over a longer period?

But the value for who? Last time I checked the consensus was that using it in any project that is not for private use can get you in legal trouble. So for me it's completely useless, even if technically it could be improved with time (which may or may not happen).

> But the value for who?

Presumably value "unlocked" for GitHub investors, by leveraging the ML model that drives some product based upon Copilot like a next-gen IDE. I agree it's not a good risk to assume for the users of Copilot.

Post reply on HN