Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI writes code, should the session be part of the commit?

github.com

Post reply on HN
Print view View original

281–290 of 410 posts

Re: If AI writes code, should the session be part of the commit?

#281

Post by veunes »

rzerowan wrote: post #122
827a wrote: post #94

IMO: This might be a contrarian opinion, but I don't think so. Its much the same problem as asking, for example, if every single line you write, or every function, becomes a commit. The answer to this granularity is, much like anything, you have to think of the audience: Who is served by persisting these sessions? I would suspect that there is little reason why future engineers, or future LLMs, would need access to t…

This is a central problem that weve already seen proliferate wildly in Scientific research , and currently if the same is allowed to be embedded in foundational code. The future outlook would be grim. Replication crisis[1]. Given initial conditions and even accounting for 'noise' would a LLm arrive at the same output.It should , for the same reason math problems require one to show their working. Scientific papers re…

Even if you pin the seed and spin up your own local LLM, changes to continuous batching at the vLLM level or just a different CUDA driver version will completely break your bitwise float convergence. Reproducibility in ML generation is a total myth, in prod we only work with the final output anyway

Re: If AI writes code, should the session be part of the commit?

#282

Post by kriro »

frank00001 wrote: post #185
jedberg wrote: post #173

The way I write code with AI is that I start with a project.md file, where I describe what I want done. I then ask it to make a plan.md file from that project.md to describe the changes it will make (or what it will create if Greenfield). I then iterate on that plan.md with the AI until it's what I want. I then ask it to make a detailed todo list from the plan.md and attach it to the end of plan.md. Once I'm fully sa…

Sounds like the spec driven approach. You should take a look at this https://github.com/github/spec-kit

I basically use a spec driven approach except I only let Github Spec Kit create the initial md file templates and then fill them myself instead of letting the agent do it. Saves a ton of tokens and is reasonably quick and I actually know I wrote the specs myself and it contains what I want. After I'm happy with the md file "harness" I let the agents loose.

The most frustrating issues that pop up are usually library/API conflicts. I work with Gymnasium or PettingZoo and Rlib or stablebaselines3. The APIs are constantly out of sync so it helps to have a working environment were libraries and APIs are in sync beforehand.

Re: If AI writes code, should the session be part of the commit?

#283

Post by matkoniecz »

jedberg wrote: post #173

The way I write code with AI is that I start with a project.md file, where I describe what I want done. I then ask it to make a plan.md file from that project.md to describe the changes it will make (or what it will create if Greenfield). I then iterate on that plan.md with the AI until it's what I want. I then ask it to make a detailed todo list from the plan.md and attach it to the end of plan.md. Once I'm fully sa…

I do the same, but put it as a comment on top of generated file.

(So far I have not used LLMs to generate code larger than fitting in one file.)

Overall idea is that I modify and tweak prompt, and keep starting new LLM sessions and dispose of old ones.

Re: If AI writes code, should the session be part of the commit?

#284

Post by voidUpdate »

criley2 wrote: post #280
lelanthran wrote: post #275

Earlier quoted context omitted.

> Prompt 1: "Research domain, think deeply, and record a full analysis in /docs/TICKET-123-NOTES.md" > Prompt 2: Based on our research, read TICKET-123 and began formulating solutions. Let's think this problem through and come up with multiple potential solutions. Document our solutions in TICKET-123-SOLUTIONS.md > Prompt 3: Based on Solution X, let's formulate a complete plan to implement. Break the work into medium…

Almost! We are certainly on the precipice of the vast majority of white collar work being removed from the loop. However, what each domain will tell you (engineering included) is that AI doesn't understand the full context of what you're doing and the point of the business and where to spend effort and where to cut corners. There is definitely still room for competent engineers to iterate here on the solutioning and…

Judging by what I've seen recently, 100% LLM code is often buggy and not that great. I'd say code quality truly matters in all domains

Re: If AI writes code, should the session be part of the commit?

#285

Post by iainmck29 »

jedberg wrote: post #173

The way I write code with AI is that I start with a project.md file, where I describe what I want done. I then ask it to make a plan.md file from that project.md to describe the changes it will make (or what it will create if Greenfield). I then iterate on that plan.md with the AI until it's what I want. I then ask it to make a detailed todo list from the plan.md and attach it to the end of plan.md. Once I'm fully sa…

is this not what entire.io is doing? Was founded by the old Github CEO Thomas Dohmke

Re: If AI writes code, should the session be part of the commit?

#286

Post by lsaferite »

slashdave wrote: post #148
solarkraft wrote: post #134

Earlier quoted context omitted.

I agree that probably not everything should be stored - it’s too noisy. But the reason the session is so interesting is precisely the later part of the conversation - all the corrections in the details, where the actual , more precise requirements crystallize.

Then just summarize the final requirements

If it's non-trivial work, have the Agent distill it down to an ADR.

Re: If AI writes code, should the session be part of the commit?

#287

Post by yuvrajangads »

The session itself is mostly noise. Half of it is the model going down wrong paths, backtracking, and trying again. Storing that alongside the commit is like saving your browser history next to your finished code.

What actually helps is a good commit message explaining the intent. If an AI wrote the code, the interesting part isn't the transcript, it's why you asked for it and what constraints you gave it. A one-paragraph description of the goal and approach is worth more than a 200-message session log.

I think the real question isn't about storing sessions, it's about whether we're writing worse commit messages because we assume the AI context is "somewhere."

Re: If AI writes code, should the session be part of the commit?

#288

Post by newswasboring »

acedTrex wrote: post #58
dang wrote: post #51

I floated that idea a week ago: https://news.ycombinator.com/item?id=47096202 , although I used the word "prompts" which users pointed out was obsolete. "Session" seems better for now. The objections I heard, which seemed solid, are (1) there's no single input to the AI (i.e. no single session or prompt) from which such a project is generated, (2) the back-and-forth between human and AI isn't exactly like working wit…

> (2) there's nothing wrong with more people being able to create and share things There is very clearly many things wrong with this when the things being shown require very little skill or effort.

Why exactly is the skill level required for something a gating parameter?

Re: If AI writes code, should the session be part of the commit?

#289

Post by criley2 »

voidUpdate wrote: post #284
criley2 wrote: post #280

Earlier quoted context omitted.

Almost! We are certainly on the precipice of the vast majority of white collar work being removed from the loop. However, what each domain will tell you (engineering included) is that AI doesn't understand the full context of what you're doing and the point of the business and where to spend effort and where to cut corners. There is definitely still room for competent engineers to iterate here on the solutioning and…

Judging by what I've seen recently, 100% LLM code is often buggy and not that great. I'd say code quality truly matters in all domains

What a utopia, where code quality matters in all domains!

In my opinion nearly the opposite is true: modern business solves for the "minimum viable quality". What is the absolute lowest quality the software can be and not tank the business.

Re: If AI writes code, should the session be part of the commit?

#290

Post by tlb »

jedberg wrote: post #173

The way I write code with AI is that I start with a project.md file, where I describe what I want done. I then ask it to make a plan.md file from that project.md to describe the changes it will make (or what it will create if Greenfield). I then iterate on that plan.md with the AI until it's what I want. I then ask it to make a detailed todo list from the plan.md and attach it to the end of plan.md. Once I'm fully sa…

Do you clear the file and use the same name for the next commit? Or create a new directory with a plan.md for each set of changes?
Post reply on HN