Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How version control will evolve for the agent boom

entire.io

Post reply on HN
Print view View original

51–60 of 72 posts

Re: How version control will evolve for the agent boom

#51

Post by staticshock »

crazygringo wrote: post #50
aabhay wrote: post #45

Earlier quoted context omitted.

Exactly. The spec should go in a git comment, NOT vcs. Or, you keep the document in history for the duration of the PR and remove it post review or pre merge

Disagree completely. The spec is vital so that future changes continue to conform to it. Specs absolutely need to live in the VCS, because they continue to be needed to keep the code conformant. They essentially are a form of code now. And code goes in the VCS.

At scale, specs can only be vital to the degree to which their conformance testing is automated. Good specs should use a formal, runnable verification language. Otherwise you'll accumulate specs that are right when they ship, wrong in subtle ways 3 months in, and wrong in glaring ways 6 months in. AI doesn't change this dynamic, it amplifies it.

Re: How version control will evolve for the agent boom

#52

Post by seanmcdirmid »

staticshock wrote: post #51
crazygringo wrote: post #50

Earlier quoted context omitted.

Disagree completely. The spec is vital so that future changes continue to conform to it. Specs absolutely need to live in the VCS, because they continue to be needed to keep the code conformant. They essentially are a form of code now. And code goes in the VCS.

At scale, specs can only be vital to the degree to which their conformance testing is automated. Good specs should use a formal, runnable verification language. Otherwise you'll accumulate specs that are right when they ship, wrong in subtle ways 3 months in, and wrong in glaring ways 6 months in. AI doesn't change this dynamic, it amplifies it.

But conformance testing is where agents really excel at if you set things up right:

* Black-box testing: the agent writing the tests cannot see implementation and the agent writing implementation cannot see tests, they only agree on a spec and an interface (the minimum needed to write tests).

* Evaluate test coverage using code coverage, but when gaps are found communicate those gaps in terms of the specification.

Good specs should be grounded (complete and not ambiguous), they don't need to be formal. You should be able to re-run your agents when the spec changes on diffs to the spec, and if a change happens out of bad, you should have agents that go in and propose fixes to the spec. Since agents are doing deterministic codegen like a compiler would, this is all pretty straightforward.

You also need to consider public and internal specifications (the public specification being for reuse of the component), and you might test your integrating component with a test-double (built from the public specification alone) rather than the real component itself.

Re: How version control will evolve for the agent boom

#53

Post by williamcotton »

sdevonoes wrote: post #42

> Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository. I don’t think this scales. We recently have been doing “spec driven development” and we are committing the specs and prompts to our repo, alongside the generated code. At the beginning it seems fine: you wanna change something, you update the spec and…

I’ve had good luck with this approach:

https://github.com/williamcotton/algraf/tree/main/docs

There’s also some tests in place to make sure some things from the master spec are up to date, eg, error codes.

Re: How version control will evolve for the agent boom

#54

Post by fsfod »

videlov wrote: post #37
linsomniac wrote: post #13

Has anyone tried teaching the agents to use jj or gitbutler-cli to do their ideas of lightweight branches? I keep feeling like that would be a huge win for the multi-branch development I'm doing much more of with agents, but I haven't yet run that experiment.

I was interested in answering this question so I built a benchmark comparing git, jj and gitbutler in agentic context https://vcbench.dev/ Disclaimer - I am a co-founder of GitButler

For Git did agents use the newer git history commands or prefer older multi step methods

Re: How version control will evolve for the agent boom

#56

Post by clintonb »

t-writescode wrote: post #40
clintonb wrote: post #20

Earlier quoted context omitted.

> Anyone else finds the LLMs version control discipline lacking? No? How do you define discipline? Claude code knows how to read and write commits. My team just granted it limited force push access a week ago. We’ve taught it to make more human-friendly commit messages. What’s missing?

I think a useful question here is “what is a good commit?”, specifically at the MR stage. I don’t care what commits look like on your local machine. I tend to like to treat commits at that stage as complete, atomic thoughts. “This commit is a refactor of this function name” “This commit defines all the data classes we’ll use” “This commit writes the new function and all its tests” “This commit injects the new code in…

I primarily care about the pull request title and description since we squash-merge and those values end up populating the squashed commit.

Re: How version control will evolve for the agent boom

#57

Post by sdevonoes »

seanmcdirmid wrote: post #52
staticshock wrote: post #51

Earlier quoted context omitted.

At scale, specs can only be vital to the degree to which their conformance testing is automated. Good specs should use a formal, runnable verification language. Otherwise you'll accumulate specs that are right when they ship, wrong in subtle ways 3 months in, and wrong in glaring ways 6 months in. AI doesn't change this dynamic, it amplifies it.

But conformance testing is where agents really excel at if you set things up right: * Black-box testing: the agent writing the tests cannot see implementation and the agent writing implementation cannot see tests, they only agree on a spec and an interface (the minimum needed to write tests). * Evaluate test coverage using code coverage, but when gaps are found communicate those gaps in terms of the specification. Go…

> Good specs should be grounded (complete and not ambiguous), they don't need to be formal

IMHO, this is a mistake. I guess we play with it because there’s isn’t anything better nowadays. Writing and maintaining “specs” in plain english is painful.

Re: How version control will evolve for the agent boom

#58

Post by t-writescode »

clintonb wrote: post #56
t-writescode wrote: post #40

Earlier quoted context omitted.

I think a useful question here is “what is a good commit?”, specifically at the MR stage. I don’t care what commits look like on your local machine. I tend to like to treat commits at that stage as complete, atomic thoughts. “This commit is a refactor of this function name” “This commit defines all the data classes we’ll use” “This commit writes the new function and all its tests” “This commit injects the new code in…

I primarily care about the pull request title and description since we squash-merge and those values end up populating the squashed commit.

What do you do to keep the cognitive load of reviewers at a minimum?

Re: How version control will evolve for the agent boom

#59

Post by seanmcdirmid »

sdevonoes wrote: post #57
seanmcdirmid wrote: post #52

Earlier quoted context omitted.

But conformance testing is where agents really excel at if you set things up right: * Black-box testing: the agent writing the tests cannot see implementation and the agent writing implementation cannot see tests, they only agree on a spec and an interface (the minimum needed to write tests). * Evaluate test coverage using code coverage, but when gaps are found communicate those gaps in terms of the specification. Go…

> Good specs should be grounded (complete and not ambiguous), they don't need to be formal IMHO, this is a mistake. I guess we play with it because there’s isn’t anything better nowadays. Writing and maintaining “specs” in plain english is painful.

Natural language is incredibly expressive and fairly easy to read and write. Pick your favorite formal specification language...and you can just express some properties in them, and they are mostly niche at that. If you increase the expressiveness of your formal language too much, it just becomes code, then you are back to square one. LLMs are also incredibly proficient in processing natural language; i.e. writing, maintaining, and using a spec written in plain english is actually fairly viable with a modern LLM.

Re: How version control will evolve for the agent boom

#60

Post by skydhash »

sdesol wrote: post #48
skydhash wrote: post #46

Earlier quoted context omitted.

> I think it will be second nature to ask "how did you arrive at the code". What about “Do you understand the code well enough to argue about it, its past, and possible evolution paths”?

I think that would be implied by "how did you arrive at the code". I think the conversation will revove around how did you "guide" things.

Not really, I don't think I've ever care about the how. I want to mostly know the trigger of a change (a bug, possible feature, or just tidying) and what are the impacts if that changes is included in the codebase. That is often just a paragraph or two.

The workflow is mostly worthless, unless it can generalize and evolve into a process. And that is more worthy of a blog post or a presentation.

Post reply on HN