Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI writes code, should the session be part of the commit?

github.com

Post reply on HN
Print view View original

221–230 of 410 posts

Re: If AI writes code, should the session be part of the commit?

#221

Post by maxbond »

dang wrote: post #51

I floated that idea a week ago: https://news.ycombinator.com/item?id=47096202 , although I used the word "prompts" which users pointed out was obsolete. "Session" seems better for now. The objections I heard, which seemed solid, are (1) there's no single input to the AI (i.e. no single session or prompt) from which such a project is generated, (2) the back-and-forth between human and AI isn't exactly like working wit…

> So, community: what should we do?

My diagnosis is that the friction that existed before (the effort to create a project) was filtering out low-effort projects and keeping the amount of submissions within the capacity the community to handle. Now that the friction is greatly reduced, there's more low-effort content and it's beyond the community's capacity (which is the real problem).

So there's two options: increase the amount of friction or increase the capacity. I don't think the capacity options are very attractive. You could add tags/categories to create different niches/queues. The most popular tags would still be overwhelmed but the more niche ones would prosper. I wouldn't mind that but I think it goes against the site's philosophy so I doubt you'll be interested.

So what I would propose is to create a heavier submission process.

- Make it so you may only submit 1 Show HN per week.

- Put it into a review queue so that it isn't immediately visible to everyone.

- Users who are eligible to be reviewers (maybe their account is at least a year old with, maybe they've posted to Show HN at least once) can volunteer to provide feedback (as comments) and can approve of the submission.

- If it gets approved by N people, it gets posted.

- If the submitter can't get the approvals they need, they can review the feedback and submit again next week.

High effort projects should sail through. Projects that aren't sufficently effortful or don't follow the Show HN guidelines (eg it's account walled) get the opportunity to apply more polish and try again.

A note on requirements for reviewers: A lot of the best comments come from people with old accounts who almost never post and so may have less than 100 karma. My interpretation is that these people have a lot of experience but only comment when they have an especially meaningful contribution. So I would suggest having requirements for account age (to make it more difficult to approve yourself from a sockpuppet) but being very flexible with karma.

Re: If AI writes code, should the session be part of the commit?

#222

Post by woctordho »

arppacket wrote: post #169
827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

While it's noisy and complicated for humans to read through, this session info is primarily for future AI to read and use as additional input for their tasks. We could have LLMs ingest all these historical sessions, and use them as context for the current session. Basically treat the current session as an extension of a much, much longer previous session. Plus, future models might be able to "understand" the limitati…

That's exactly one of the reasons I've been archiving the sessions using DataClaw. The sessions can contain more useful information than the comments for humans.

[0] https://github.com/peteromallet/dataclaw

Re: If AI writes code, should the session be part of the commit?

#223

Post by woctordho »

panarky wrote: post #180
eru wrote: post #111

Earlier quoted context omitted.

> Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. Hmm, I think that's the wrong comparison? The more useful comparison might be: should all your notes you made and dead ends you tried become part of the commit?

When a human writes the code should all their slack messages about the project be committed into the repo?

Ideally, yes. Although Slack is a vendor lock-in and we need a better platform to archive the sessions.

Re: If AI writes code, should the session be part of the commit?

#224

Post by jedberg »

fhub wrote: post #220
jedberg wrote: post #173

The way I write code with AI is that I start with a project.md file, where I describe what I want done. I then ask it to make a plan.md file from that project.md to describe the changes it will make (or what it will create if Greenfield). I then iterate on that plan.md with the AI until it's what I want. I then ask it to make a detailed todo list from the plan.md and attach it to the end of plan.md. Once I'm fully sa…

I do something similar but I get Claude to review Codex every step of the way and feed it back (or visa versa depending on day)

My next step was to add in having another LLM review Claude's plans. With a few markdown artifacts it should be easy for the other LLM to figure it out and make suggestions.

Re: If AI writes code, should the session be part of the commit?

#226

Post by xigoi »

tpmoney wrote: post #110
D-Machine wrote: post #96

Earlier quoted context omitted.

It should be a distillation of the session and/or the prompts, at bare minimum. No, it should not include e.g. research-type questions, but it should include prompts that the user wrote after reading the answers to those research-type questions, and perhaps some distillation of the links / references surfaced during the research. Prompts probably should be distilled / summarized, especially if they are research-based…

You have the source code though. That is the "reproducibility" bit you need. What extra reproducibility does having the prompts give you? Especially given that AI agents are non-deterministic in the first place. To me the idea that the prompts and sessions should be part of the commit history is akin to saying that the keystroke logs and commands issued to the IDE should be part of the commit history. Is it important…

The source code is whatever is easiest for a human to understand. Committing AI-generated code without the prompts is like committing compiler-generated machine code.

Re: If AI writes code, should the session be part of the commit?

#227

Post by woctordho »

sillysaurusx wrote: post #61
dang wrote: post #51

I floated that idea a week ago: https://news.ycombinator.com/item?id=47096202 , although I used the word "prompts" which users pointed out was obsolete. "Session" seems better for now. The objections I heard, which seemed solid, are (1) there's no single input to the AI (i.e. no single session or prompt) from which such a project is generated, (2) the back-and-forth between human and AI isn't exactly like working wit…

Unfortunately Codex doesn’t seem to be able to export the entire session as markdown, otherwise I’d suggest encouraging people to include that in their Show HNs. It’s kind of nuts that it’s so difficult to export what’s now a part of the engineering process. I don’t have anything against vibe coded apps, but what makes them interesting is to see the vibe coding session and all the false starts along the way. You lear…

You can export it with DataClaw. By default it outputs jsonl and publishes to HuggingFace, but you can also do analysis locally with it.

Re: If AI writes code, should the session be part of the commit?

#228

Post by jumploops »

jedberg wrote: post #219
jumploops wrote: post #205

Earlier quoted context omitted.

I do something similar, but across three doc types: design, plan, and debug Design works similar to your project.md file, but on a per feature request. I also explicitly ask it to outline open questions/unknowns. Once the design doc (i.e. design/[feature].md) has been sufficiently iterated on, we move to the plan doc(s). The plan docs are structured like `plan/[feature]/phase-N-[description].md` From here, the agent…

> At this point, I either jump back to new design/plan files, or dive into the debug flow. Similar to the plan prompting, debug is instructed to review the current implementation, and outline N-M hypotheses for what could be wrong. I'm biased because my company makes a durable execution library, but I'm super excited about the debug workflow we recently enabled when we launched both a skill and MCP server. You can us…

This is great, giving agents access to logs (dev or prod) tightens the debug flow substantially.

With that said, I often find myself leaning on the debug flow for non-errors e.g. UI/UX regressions that the models are still bad at visualizing.

As an example, I added a "SlopGoo" component to a side project, which uses an animated SVG to produce a "goo" like effect. Ended up going through 8 debug docs[0] until I was satisified.

[0]https://github.com/jumploops/slop.haus/tree/main/debug

Re: If AI writes code, should the session be part of the commit?

#229

Post by woctordho »

YoumuChan wrote: post #27

Should my google search history be part of the commit? To that question my answer is no.

If you archive the session, you automatically archive all Google search history (queries and outputs) that the AI did, and it's usually relevant to the project.
Post reply on HN