Gptcommit: Never write a commit message again (with the help of GPT-3)

Post by bee_rider » Fri, Jan 20, 2023, 1:13 AM UTC

TeMPOraL wrote: post #37

dheera wrote: post #20

Earlier quoted context omitted.

Except for startups when commit messages are more like "asdf", "aoeu", "quick fix", or "demo" because some investor barged in and demanded a demo before they would wire funds. If ChatGPT could change that to something like "disable current limits" or "disable safety checks" or whatever that might be marginally better.

A ChatGPT-generated message, pasted without editing, is purely functional transformation of the code, adding zero information. This means I could just as well run it on your diff myself , if I thought it would be useful. More than that, when I do it a year or two after you made your commit, the then-current ChatGPT will likely do a much better job at summarizing the change. So perhaps it's best to leave auto-summariz…

Post by dheera » Fri, Jan 20, 2023, 1:31 AM UTC

TeMPOraL wrote: post #55

dheera wrote: post #40

Earlier quoted context omitted.

> purely functional transformation of the code, adding zero information I mean, a human brain is arguably also a purely functional transformation adding zero information.

It's not. The diff, which is the sole input to GPT-3 here, does not carry the causal context - that is, why the change was made. Nor does it carry the broader understanding necessary to summarize the what succinctly and accurately - things like high-level design terms that mostly exist on diagrams or in issue trackers, but not in the code itself. By adding those details in a commit message, the author can add extra i…

Post by RheingoldRiver » Fri, Jan 20, 2023, 1:51 AM UTC

throwanem wrote: post #42

tomtomtom777 wrote: post #31

Earlier quoted context omitted.

For me the point of this demo is that even a good commit message is often redundant information. As programmers we learn that adding a comment like: // The limit is 16 to const SOME_LIMIT = 16 is bad because is redundant information that serves no purpose to the reader and can easily misalign in the future. So what's a good commit message for changing this limit? Ideally we want to describe why we've changed it but t…

If you don't know why you're making the change, you are not ready to commit the change.

Post by hgsgm » Fri, Jan 20, 2023, 2:04 AM UTC

ketralnis wrote: post #18

I don't really ever want to read answers from GPT to questions that I didn't knowingly myself ask GPT. If GPT can write a commit message from you, don't write it at all and let me ask it that if that's what I want. It may be a positive to you to spend a few seconds less on commit messages but it's a net negative for the world for it to become polluted with vast amounts of flatly incorrect text with no knowledge as to…

Post by hgsgm » Fri, Jan 20, 2023, 2:10 AM UTC

TeMPOraL wrote: post #32

jart wrote: post #17

Earlier quoted context omitted.

The point of a summarization model is if you have a thousand line change, it helps to have a one sentence explanation of what it is. The demo videos the author used here really don't do a good job communicating that, because the summary GPT-3 wrote for his one line commit was longer than the commit itself.

Right, and even if GPT-3 could summarize the thousand-line diff in a sensible way, without introducing any falsehoods, it would still be strictly worse than the developer writing a sentence explaining what they think they've accomplished with the commit. It's just the same thing as with comments and "self-documenting code". The code tells you what (and if written carefully, it may be even somewhat effective at it). I…