Should your browser and search history be part of the commit too?
If AI writes code, should the session be part of the commit?
341–350 of 410 posts
Re: If AI writes code, should the session be part of the commit?
#342LLMs are non-deterministic, so feeding that session back in possibly will get you a different output. Also, models change over time so you may not necessarily be able to run the session against the same model again
The whole point of the source code it generates is to have the artifact. Maybe this is somewhat useful if you need to train people how to use AI, but at the end of the day the generated code is the thing that matters. If you keep other notes/documentation from meetings and design sessions, however you keep that is probably where this should go, too?
Re: If AI writes code, should the session be part of the commit?
#343Earlier quoted context omitted.
It's exhaust. Retrospectively, chat is essentially worthless. You're going to chase hallucinations down conversations that maybe didn't even impact code.
I have difficulty believing chat is worthless. And I think that not everyone will entertain or chase the hallucinations down. Or maybe enough non-hallucinations are chased that it is valuable.
You seem to be thinking like it is 2020 and humans will be the ones reading the chat. It is just context-bloat for whatever agent ends up reading it.
Re: If AI writes code, should the session be part of the commit?
#344One of the use cases i see for this tool is helping companies to understand the output coming from the llm blackbox and the process which the employee took to complete a certain task
Except it doesn't capture the majority of uses of AI, in my experience. In my current practice, the the vast majority of AI use is autocompletions, or small inline prompts. ("Fix this error."; "Open an ALSA midi connection" (things that avoid a to trip into awful documentation); "if (one of the query parameters is "gear='ir') ..." (things that break flow by forcing a trip into excellent but overly verbose Javascript…
What coding agent are you using at your workplace?
Re: If AI writes code, should the session be part of the commit?
#345The way I write code with AI is that I start with a project.md file, where I describe what I want done. I then ask it to make a plan.md file from that project.md to describe the changes it will make (or what it will create if Greenfield). I then iterate on that plan.md with the AI until it's what I want. I then ask it to make a detailed todo list from the plan.md and attach it to the end of plan.md. Once I'm fully sa…
Sounds like the spec driven approach. You should take a look at this https://github.com/github/spec-kit
or OpenSpec https://github.com/Fission-AI/OpenSpec/
I think it's much better
Re: If AI writes code, should the session be part of the commit?
#346Since the code is literally the answer to What comes next after this prompt the answer is yes. Unfortunately there is also a hidden random seed in the engine (which this doesn't seem to address). But if you capture the seed, the exact version of the software and the prompt, the system is completely deterministic.
However there is an unpleasant reality: the system could be incredibly brittle, with the slightest change in input or seed resulting in significantly different output. It would be nice if all small and seemingly inconsequential input perturbations resulted in a cluster of outputs that are more or less the same, but that seems very model dependent.
Re: If AI writes code, should the session be part of the commit?
#347IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…
While it's noisy and complicated for humans to read through, this session info is primarily for future AI to read and use as additional input for their tasks. We could have LLMs ingest all these historical sessions, and use them as context for the current session. Basically treat the current session as an extension of a much, much longer previous session. Plus, future models might be able to "understand" the limitati…
It's just noise for AI too. There is no reason to be lazy with context management when you can simply ask the AI to write the summary of the session. But even that is hardly useful when AI can just read the source of truth which is the code and committed docs
Re: If AI writes code, should the session be part of the commit?
#348LLMs are non-deterministic, so feeding that session back in possibly will get you a different output. Also, models change over time so you may not necessarily be able to run the session against the same model again The whole point of the source code it generates is to have the artifact. Maybe this is somewhat useful if you need to train people how to use AI, but at the end of the day the generated code is the thing t…
They are completely deterministic? We introduce pseudo-randomness to assist with exploring the solution space.
The fact that models change is a great reason to be able to re-run a previous model and maintain revision control and repeatability.
The source code artifact is not really the point. Not anymore.
Re: If AI writes code, should the session be part of the commit?
#349Should your browser and search history be part of the commit too?
The thought you have while coding should be part of your workbook. This is a distillation of all the input and processing at the time, which can be a valuable clue for bug hunting and refactoring.
Re: If AI writes code, should the session be part of the commit?
#350The way I write code with AI is that I start with a project.md file, where I describe what I want done. I then ask it to make a plan.md file from that project.md to describe the changes it will make (or what it will create if Greenfield). I then iterate on that plan.md with the AI until it's what I want. I then ask it to make a detailed todo list from the plan.md and attach it to the end of plan.md. Once I'm fully sa…
I do something similar, but across three doc types: design, plan, and debug Design works similar to your project.md file, but on a per feature request. I also explicitly ask it to outline open questions/unknowns. Once the design doc (i.e. design/[feature].md) has been sufficiently iterated on, we move to the plan doc(s). The plan docs are structured like `plan/[feature]/phase-N-[description].md` From here, the agent…
I have a similar process and have thought about committing all the planning files, but I've found that they tend to end up in an outdated state by the time the implementation is done.
Better imo is to produce a README or dev-facing doc at the end that distills all the planning and implementation into a final authoritative overview. This is easier for both humans and agents to digest than bunch of meandering planning files.