Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI writes code, should the session be part of the commit?

github.com

Post reply on HN
Print view View original

161–170 of 410 posts

Re: If AI writes code, should the session be part of the commit?

#161

Post by Muromec »

827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

There is some potential value for the audit if you work in a special place where you are sworn in and where transparency is important, but who gonna read all of that and how do you even know that the transcript corresponds to the code if the committer is up to something

Re: If AI writes code, should the session be part of the commit?

#163

Post by [deleted] »

dang wrote: post #51

I floated that idea a week ago: https://news.ycombinator.com/item?id=47096202 , although I used the word "prompts" which users pointed out was obsolete. "Session" seems better for now. The objections I heard, which seemed solid, are (1) there's no single input to the AI (i.e. no single session or prompt) from which such a project is generated, (2) the back-and-forth between human and AI isn't exactly like working wit…

[deleted]

Re: If AI writes code, should the session be part of the commit?

#164

Post by globular-toast »

Like any discussion about AI there are two things people are talking about here and it's not always clear which:

1. Using LLMs as a tool but still very much crafting the software "by hand",

2. Just prompting LLMs, not reading or understanding the source code and just running the software to verify the output.

A lot of comments here seem to be thinking of 1. But I'm pretty sure the OP is thinking of 2.

Re: If AI writes code, should the session be part of the commit?

#166

Post by brainlounge »

The more fundamental question is: Is there information in the AI-coding session that should be preserved? Only if the answer is "yes", the next question becomes: Where do we store that data?

git is only one possible location.

I think there is very valuable information in session logs, like the prompts, or the usage statistics at the end of the session, which model was used etc. But git history or the commit messages should focus on the outcome of the work, not on the process itself. This is why the whole issue discussion before work in git starts is also typically kept separately in tickets. Not in git itself, but close to it.

There're platforms like tulpal.com which move the whole local agent-supported process to the server and therefore have much better after-the-fact observability in what happened.

Re: If AI writes code, should the session be part of the commit?

#167

Post by AlexCoventry »

majormajor wrote: post #145
rzerowan wrote: post #122

Earlier quoted context omitted.

This is a central problem that weve already seen proliferate wildly in Scientific research , and currently if the same is allowed to be embedded in foundational code. The future outlook would be grim. Replication crisis[1]. Given initial conditions and even accounting for 'noise' would a LLm arrive at the same output.It should , for the same reason math problems require one to show their working. Scientific papers re…

You can leave commit messages or comments without spamming your history with every "now I'm inspecting this file..." or "oops, that actually works differently than I expected" transcript. In fact, I'd wager that all that excess noise would make it harder to discern meaningful things in the future than simply distilling the meaningful parts of the session into comments and commit messages.

IMO, you should do both. The cost of intellectual effort is dropping to zero, and getting an AI to scan through a transcript for relevant details is not going to cost much at all.

Re: If AI writes code, should the session be part of the commit?

#169

Post by arppacket »

827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

While it's noisy and complicated for humans to read through, this session info is primarily for future AI to read and use as additional input for their tasks.

We could have LLMs ingest all these historical sessions, and use them as context for the current session. Basically treat the current session as an extension of a much, much longer previous session.

Plus, future models might be able to "understand" the limitations of current models, and use the historical session info to identity where the generated code could have deviated from user intention. That might be useful for generating code, or just more efficient analysis by focusing on possible "hotspots", etc.

Basically, it's high time we start capturing any and all human input for future models, especially open source model development, because I'm sure the companies already have a bunch of this kind of data.

Re: If AI writes code, should the session be part of the commit?

#170

Post by imiric »

dang wrote: post #64
acedTrex wrote: post #58

Earlier quoted context omitted.

> (2) there's nothing wrong with more people being able to create and share things There is very clearly many things wrong with this when the things being shown require very little skill or effort.

That is by no means all of these projects. I'm not interested in a circle-the-wagons crackdown because it won't work (see "it's foolish to fight the future" above), and because we should be welcoming and educating new users in how to contribute substantively to HN.

Which users?

The future you're concerned with defending includes bots being a large part of this community, potentially the majority. Those bots will not only submit comments autonomously, but create these projects, and Show HN threads. I.e. there will be no human in the loop.

This is not something unique to this forum, but to the internet at large. We're drowning in bot-generated content, and now it is fully automated.

So the fundamental question is: do you want to treat bots as human users?

Ignoring the existential issue, whatever answer you choose, it will inevitably alienate a portion of existing (human) users. It's silly I have to say this, but bots don't think, nor "care", and will keep coming regardless.

To me the obvious answer is "no". All web sites that wish to preserve their humanity will have to do a complete block of machine-generated content, or, at the very least, filter and categorize it correctly so that humans who wish to ignore it, can. It's a tough nut to crack, but I reckon YC would know some people capable of tackling this.

It's important to note that this state of a human driving the machine directly is only temporary. The people who think these are tools as any other are sorely mistaken. This tool can do their minimal effort job much more efficiently, cheaper, and with better results, and it's only a matter of time until the human is completely displaced. This will take longer for more complex work, of course, but creating regurgitated projects on GitHub and posting content on discussion forums is a very low bar activity.

Post reply on HN