Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI writes code, should the session be part of the commit?

github.com

Post reply on HN
Print view View original

121–130 of 410 posts

Re: If AI writes code, should the session be part of the commit?

#121

Post by mandel_x »

dang wrote: post #51

I floated that idea a week ago: https://news.ycombinator.com/item?id=47096202 , although I used the word "prompts" which users pointed out was obsolete. "Session" seems better for now. The objections I heard, which seemed solid, are (1) there's no single input to the AI (i.e. no single session or prompt) from which such a project is generated, (2) the back-and-forth between human and AI isn't exactly like working wit…

> people have been submitting so many Show HNs of generated projects

In this case, it was more of write the X language compiler using X. I had to prove to myself if keeping the session made sense, and what better way to do it than to vibe code the tool to audit vibe code.

I do get your point though

Re: If AI writes code, should the session be part of the commit?

#122

Post by rzerowan »

827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

This is a central problem that weve already seen proliferate wildly in Scientific research , and currently if the same is allowed to be embedded in foundational code. The future outlook would be grim.

Replication crisis[1].

Given initial conditions and even accounting for 'noise' would a LLm arrive at the same output.It should , for the same reason math problems require one to show their working. Scientific papers require the methods and pseudocode while also requireing limitations to be stated.

Without similar guardrails , maintainance and extension of future code becomes a choose your own adventure.Where you have to guess at the intent and conditions of the LLM used.

[1] https://www.ipr.northwestern.edu/news/2024/an-existential-cr...

Re: If AI writes code, should the session be part of the commit?

#123

Post by brendanmc6 »

eru wrote: post #114
brendanmc6 wrote: post #107

A few things really leveled up both my software quality and my productivity in the last few months. It wasn’t session history, memory files, context management or any of that. 1. Writing a spec with clear acceptance criteria. 2. Assigning IDs to my acceptance criteria. Sounds tedious, but actually the idea wasn’t mine, at some point an agent went and did it without me asking. The references proved so useful for guidi…

> Assigning IDs to my acceptance criteria. Sounds tedious, [...] Might be tedious for a human, but agents should do that just fine?

Yup and they do, but then I figured out that I can just write loosely structured yaml and the ids come for free. I then encourage the agents to tag and reference them everywhere, especially tests.

Re: If AI writes code, should the session be part of the commit?

#124

Post by notedbrew »

827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

You ignore the reality of vibe coding. If someone just prompts and never reads the code and tests the result barely, then the prompts can be a valuable insight.

But I am not rooting for either, just saying.

Re: If AI writes code, should the session be part of the commit?

#125

Post by D-Machine »

tpmoney wrote: post #110
D-Machine wrote: post #96

Earlier quoted context omitted.

It should be a distillation of the session and/or the prompts, at bare minimum. No, it should not include e.g. research-type questions, but it should include prompts that the user wrote after reading the answers to those research-type questions, and perhaps some distillation of the links / references surfaced during the research. Prompts probably should be distilled / summarized, especially if they are research-based…

You have the source code though. That is the "reproducibility" bit you need. What extra reproducibility does having the prompts give you? Especially given that AI agents are non-deterministic in the first place. To me the idea that the prompts and sessions should be part of the commit history is akin to saying that the keystroke logs and commands issued to the IDE should be part of the commit history. Is it important…

> You have the source code though. That is the "reproducibility" bit you need.

I am talking about reproducing the (perhaps erroneous) logic or thinking or motivations in cases of bugs, not reproducing outputs perfectly. As you said, current LLM models are non-deterministic, so we can't have perfect reproducibility based on the prompts, but, when trying to fix a bug, having the basic prompts we can see if we run into similar issues given a bad prompt. This gives us information about whether the bad / bugged code was just a random spasm, or something reflecting bad / missing logic in the prompt.

> Is it important to know that when the foo file was refactored the developer chose to do it by hand vs letting the IDE do it with an auto-refactor command vs just doing a simple find and replace? Maybe it is for code review purposes, but for "reproducibility" I don't think it is.

I am really using "reproducibility" more abstractly here, and don't mean perfect reproducibility of the same code. I.e. consider this situation: "A developer said AI wrote this code according to these specs and prompt, which, according to all reviewers, shouldn't produce the errors and bad code we are seeing. Let's see if we can indeed reproduce similar code given their specs and prompt". The less evidence we have of the specifics of a session, the less reproducible their generated code is, in this sense.

Re: If AI writes code, should the session be part of the commit?

#127

Post by D-Machine »

e3bc54b2 wrote: post #120
D-Machine wrote: post #96

Earlier quoted context omitted.

It should be a distillation of the session and/or the prompts, at bare minimum. No, it should not include e.g. research-type questions, but it should include prompts that the user wrote after reading the answers to those research-type questions, and perhaps some distillation of the links / references surfaced during the research. Prompts probably should be distilled / summarized, especially if they are research-based…

> It should be a distillation of the session and/or the prompts, at bare minimum. Huh, I thought that's what commit message is for.

I mean, sure, a good, detailed commit message is perfectly fine to me in place of the prompts / a session distillation. But I am not holding my breath for vibe-coders to properly review their code and make such a commit message. But, if they, do, great! No need for prompt / session details.

Re: If AI writes code, should the session be part of the commit?

#129

Post by [deleted] »

grayhatter wrote: post #100
dang wrote: post #51

I floated that idea a week ago: https://news.ycombinator.com/item?id=47096202 , although I used the word "prompts" which users pointed out was obsolete. "Session" seems better for now. The objections I heard, which seemed solid, are (1) there's no single input to the AI (i.e. no single session or prompt) from which such a project is generated, (2) the back-and-forth between human and AI isn't exactly like working wit…

> So, community: what should we do? > Is Show HN dead? No, but it's drowning Is spam on topic? and are AI codegen bots part of the community? To me, the value of Show HN was rarely the thing , it was the work and attention that someone put into it. AI bot's don't do work. (What they do is worth it's own word, but it's not the same as work). > I don't want to exclude these projects, because (1) some of them are good,…

[deleted]

Re: If AI writes code, should the session be part of the commit?

#130

Post by 827a »

rzerowan wrote: post #122
827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

This is a central problem that weve already seen proliferate wildly in Scientific research , and currently if the same is allowed to be embedded in foundational code. The future outlook would be grim. Replication crisis[1]. Given initial conditions and even accounting for 'noise' would a LLm arrive at the same output.It should , for the same reason math problems require one to show their working. Scientific papers re…

Agentic engineering is fundamentally different, not just because of the inherent unpredictability of LLMs, but also because there's a wildly good chance that two years from now Opus 4.6 will no longer even be a model anyone can use to write code with.
Post reply on HN