Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI writes code, should the session be part of the commit?

github.com

Post reply on HN
Print view View original

101–110 of 410 posts

Re: If AI writes code, should the session be part of the commit?

#101

Post by D-Machine »

alainrk wrote: post #95

My complete reasoning, notes, errors have never been part of the commit. I don't see a valid reason on why the raw conversation must be included. Rather I have hooks (or just "manually" invoked) to process all of it and update the relevant documentation that I've been putting under docs/.

If you also ensure the AI writes relevant (and correct) docs, and also code comments and commit message, then I agree there is not much need for extra info, e.g. prompts / session distillation. I am not sure that that is the case currently (though we might be getting there soon at least in some cases).

Re: If AI writes code, should the session be part of the commit?

#102

Post by stackghost »

827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

LLM session transcripts as part of the commit is a neat idea to consider, to be sure, but I know that I damn well don't want to read eight pages of "You're absolutely right! It's not a foo. It's a bar" slop (for each commit no less!) when I'm trying to find someone to git blame.

The solution is as it always has been: the commit message is where you convey to your fellow humans, succinctly and clearly, why you made the commit.

I like the idea of committing the initial transcript somewhere in the docs/ directory or something. I'll very likely start doing this in my side projects.

Re: If AI writes code, should the session be part of the commit?

#103

Post by fladrif »

I think this is a lot of "kicking can down the road" of not understanding what code the ai is writing. Once you give up understanding the code that is written there is no going back. You can add all the helper commit messages, architecture designs, plans, but then you introduce the problem of having to read all of those once you run into an issue. We've left readability on the wayside to the alter of "writeability".

The paradigm shift, which is a shift back, is to embrace the fact that you have to slow down, and understand all the code the ai is writing.

Re: If AI writes code, should the session be part of the commit?

#104

Post by onion2k »

Conceptually this is very similar to the question of whether or not you should squash your commits. To the point that it's really the same question.

If you think you should squash commits, then you're only really interested in the final code change. The history of how the dev got there can go in the bin.

If you don't think you should squash commits then you're interested in being able to look back at the journey that got the dev to the final code change.

Both approaches are valid for different reasons but they're a source of long and furious debate on every team I've been on. Whether or not you should be keeping a history of your AI sessions alongside the code could be useful for debugging (less code debugging, more thought process debugging) but the 'prefer squash' developers usually prefer to look the existing code rather than the history of changes to steer it back on course, so why would they start looking at AI sessions if they don't look at commits?

All that said, your AI's memory could easily be stored and managed somewhere separately to the repo history, and in a way that makes it more easily accessible to the LLM you choose, so probably not.

Re: If AI writes code, should the session be part of the commit?

#105

Post by hakanderyal »

I created a system which I call 'devlog'. Agent summarizes what it did & how it did in a concise file, and its gets committed along with first prompt and the plan file if any. Later due to noise & volume, I started saving those in a database and adding only devlog id to commit nowadays.

Now whenever I need to reason with what agent did & why, info is linked & ready on demand. If needed, session is also saved.

It helps a lot.

Re: If AI writes code, should the session be part of the commit?

#106

Post by mandel_x »

wging wrote: post #93
dang wrote: post #51

I floated that idea a week ago: https://news.ycombinator.com/item?id=47096202 , although I used the word "prompts" which users pointed out was obsolete. "Session" seems better for now. The objections I heard, which seemed solid, are (1) there's no single input to the AI (i.e. no single session or prompt) from which such a project is generated, (2) the back-and-forth between human and AI isn't exactly like working wit…

Regarding the noise you mention, I wonder if memento's use of the git 'notes' feature is an acceptable way to contain or quarantine that noise. It might still not add much value, but at least it would live in a separate place that is easily filtered out when the user judges it irrelevant. Per the README of the linked repo, > It runs a commit and then stores a cleaned markdown conversation as a git note on the new com…

You are 100% and that’s why I chose git notes. If you do not sync them you have no knowledge of their existence.

Re: If AI writes code, should the session be part of the commit?

#107

Post by brendanmc6 »

A few things really leveled up both my software quality and my productivity in the last few months. It wasn’t session history, memory files, context management or any of that.

1. Writing a spec with clear acceptance criteria.

2. Assigning IDs to my acceptance criteria. Sounds tedious, but actually the idea wasn’t mine, at some point an agent went and did it without me asking. The references proved so useful for guiding my review that I formalized the process (and switched from .md to .yaml to make it easier).

3. Giving my agents a source of truth to share implementation progress so they can plan their own tasks and more effectively review.

Of course, I can’t help myself, I had to formalize it into a spec standard and a toolkit. Gonna open source it all soon, but I really want feedback before I go too far down the rabbit hole:

https://acai.sh

Re: If AI writes code, should the session be part of the commit?

#108

Post by D-Machine »

onion2k wrote: post #104

Conceptually this is very similar to the question of whether or not you should squash your commits. To the point that it's really the same question. If you think you should squash commits, then you're only really interested in the final code change. The history of how the dev got there can go in the bin. If you don't think you should squash commits then you're interested in being able to look back at the journey that…

I think this is the right analogy, contrary to some other very poor ones in this thread. Yes, it is rare to really look at commit messages, but it can be invaluable in some cases.

With vibe-coding, you risk having no documentation at all for the reasoning (AI comments and tests can be degenerate / useless), but the prompts, at bare minimum, reveal something about the reasoning / motivation.

Whether this needs to be in git or not is a side issue, but there is benefit to having this available.

Re: If AI writes code, should the session be part of the commit?

#109

Post by mikepurvis »

onion2k wrote: post #104

Conceptually this is very similar to the question of whether or not you should squash your commits. To the point that it's really the same question. If you think you should squash commits, then you're only really interested in the final code change. The history of how the dev got there can go in the bin. If you don't think you should squash commits then you're interested in being able to look back at the journey that…

I've generally been in the squash camp but it's more out of a sense of wanting a "clean" and bisectable repo history. In a word where git (and git forges) could show me atomic merge commits but also let me seamlessly fan those out to show the internal history and iteration and maybe stuff like llm sessions, I'd be into that.

And yes, it's my understanding that mercurial and fossil do actually do more of this than git does, but I haven't actually worked on any projects using those so I can't comment.

Re: If AI writes code, should the session be part of the commit?

#110

Post by tpmoney »

D-Machine wrote: post #96
rfw300 wrote: post #14

Why should it be? The agent session is a messy intermediate output, not an artifact that should be part of the final product. If the "why" of a code change is important, have your agent write a commit message or a documentation file that is polished and intended for consumption.

It should be a distillation of the session and/or the prompts, at bare minimum. No, it should not include e.g. research-type questions, but it should include prompts that the user wrote after reading the answers to those research-type questions, and perhaps some distillation of the links / references surfaced during the research. Prompts probably should be distilled / summarized, especially if they are research-based…

You have the source code though. That is the "reproducibility" bit you need. What extra reproducibility does having the prompts give you? Especially given that AI agents are non-deterministic in the first place. To me the idea that the prompts and sessions should be part of the commit history is akin to saying that the keystroke logs and commands issued to the IDE should be part of the commit history. Is it important to know that when the foo file was refactored the developer chose to do it by hand vs letting the IDE do it with an auto-refactor command vs just doing a simple find and replace? Maybe it is for code review purposes, but for "reproducibility" I don't think it is. You have the code that made build X and you have the code that made build X+1. As long as you can reliably recreate X and X+1 from what you have in the code, you have reproducibility.
Post reply on HN