Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents that run while I sleep

claudecodecamp.com

Post reply on HN
Print view View original

171–180 of 532 posts

Re: Agents that run while I sleep

#171

Post by hinkley »

wnevets wrote: post #33
jdlshore wrote: post #27

Pet peeve: this post misunderstands “TDD.” What it really describes is acceptance tests. TDD is a tool for working in small steps, so you get continuous feedback on your work as you go, and so you can refine your design based on how easy it is to use in practice. It’s “red green refactor repeat”, and each step is only a handful of lines of code. TDD is not “write the tests, then write the code.” It’s “write the tests…

> TDD is a tool for working in small steps, so you get continuous feedback on your work as you go, and so you can refine your design based on how easy it is to use in practice. I would like to emphasize that feedback includes being alerted to breaking something you previously had working in a seemly unrelated/impossible way.

Accidentally mutating an input is always a 'fun' way to trigger spooky action at a distance.

Re: Agents that run while I sleep

#172

Post by mr-wendel »

what wrote: post #120
jeremyjh wrote: post #110

Earlier quoted context omitted.

What is strange about paying for tools that improve productivity? Unless you consider your own time worthless you should always be open to spending more to gain more.

Your own time is worthless if you’re not spending it doing something that makes more money. You don’t make more money increasing your productivity for work when you’re expected to work the same number of hours.

I've spent a fair amount of time contracting -- this issue is even more relevant here. While I wasn't spending very much on AI tools, what I did spent was worth every penny... for the company I was supporting :).

Fortunately, there was enough work to be done so productivity increases didn't decrease my billable hours. Even if it did, I still would have done it. If it helps me help others, then it's good for my reputation. Thats hard to put a price on, but absolutely worth what I paid in this case.

Re: Agents that run while I sleep

#174

Post by throwatdem12311 »

storus wrote: post #50

Wasn't the best practice to run one model/coding agent that writes the code and another one that reviews it? E.g. Claude Code for writing the code, GPT Codex to review/critique it? Different reward functions.

It’s superstition that using a different slop generator to “review” the slop from a different brand of slop generator somehow makes things better. It’s slop all the way down.

Re: Agents that run while I sleep

#175

Post by hexaga »

lagrange77 wrote: post #151
hexaga wrote: post #92

Earlier quoted context omitted.

There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model. And with that comes reward hacking - which isn't really about looking for more reward but rather that the model has learned patterns of behavior that got reward in the train env. That is, any kind of vulnerability in the train env manifests as something you'd recognize as reward hacking in the real world:…

> There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model. Hm, as i understand it, parts of the training of e.g. ChatGPT could be called RL models. But the subject to be trained/fine tuned is still a seq2seq next token predictor transformer neural net.

RL is simply a broad category of training methods. It's not really an architecture per se: modern GPTs are trained first on reconstruction objective on massive text corpora (the 'large language' part), then on various RL objectives +/- more post-training depending on which lab.

Re: Agents that run while I sleep

#176

Post by throwaway7783 »

Regarding the self-congratulation machine - I simply use a different claude code session to do the reviews. There is no self-congratulation, but overly critical at times. Works well.

Honestly, sometimes the harnesses, specs, some predefined structure for skills etc all feel over-engineering. 99% of the time a bloody prompt will do. Claude Code is capable of planning, spawning sub-agents, writing tests and so on.

Claude.md file with general guidelines about our repo has worked extraordinarily good, without any external wrappers, harnesses or special prompts. Even the MD file has no specific structure, just instructions or notes in English.

Re: Agents that run while I sleep

#177

Post by throwaway7783 »

w4yai wrote: post #117
egeozcan wrote: post #65

Earlier quoted context omitted.

The trick is, with the setup I mentioned, you change the rewards. The concept is: Red Team (Test Writers), write tests without seeing implementation. They define what the code should do based on specs/requirements only. Rewarded by test failures. A new test that passes immediately is suspicious as it means either the implementation already covers it (diminishing returns) or the test is tautological. Red's ideal outco…

You guys are describing wonderful things, but I've yet to see any implementation. I tried coding my own agents, yet the results were disappointing. What kind of setup do you use ? Can you share ? How much does it cost ?

We have a very uncomplicated setup with claude code. A CLAUDE.md with instructions and notes about the repo and how to run stuff. We also do code reviews with Claude Code, but in a separate session.

It works wonderfully well. Costs about $200USD per developer per month as of now.

Re: Agents that run while I sleep

#178

Post by canadiantim »

nojito wrote: post #157
canadiantim wrote: post #135

Earlier quoted context omitted.

Check out Mike Pocock’s work, he’s done excellent work writing about red green refactor and has a GitHub repo for his skills. Read and take what you need from his tdd skill and incorporate it into your own tdd skill tailored for your project.

This is just ai slop. If you follow what the actual designers of Claude/GPT tell you it flys in the face of building out over engineered harnesses for agents.

Works better than standard claude / gpt, which doesn't do red-green-refactor. Doesn't seem like slop when it meaningfully changes the results for the better, consistently. Really is a game-changer. You should consider trying it.

Re: Agents that run while I sleep

#179

Post by throwaway7783 »

nojito wrote: post #157
canadiantim wrote: post #135

Earlier quoted context omitted.

Check out Mike Pocock’s work, he’s done excellent work writing about red green refactor and has a GitHub repo for his skills. Read and take what you need from his tdd skill and incorporate it into your own tdd skill tailored for your project.

This is just ai slop. If you follow what the actual designers of Claude/GPT tell you it flys in the face of building out over engineered harnesses for agents.

I agree with this. There is not a lot of harnesses/wrapping needed for Claude Code.

Re: Agents that run while I sleep

#180

Post by throwaway7783 »

tomtom1337 wrote: post #70
egeozcan wrote: post #65

Earlier quoted context omitted.

The trick is, with the setup I mentioned, you change the rewards. The concept is: Red Team (Test Writers), write tests without seeing implementation. They define what the code should do based on specs/requirements only. Rewarded by test failures. A new test that passes immediately is suspicious as it means either the implementation already covers it (diminishing returns) or the test is tautological. Red's ideal outco…

This is very interesting, but like sibling comments, I'm very curious as to how you run this in practice. Do you just tell Claude/Copilot to do what you describe? And do you have any prompts to share?

You don't need most of this. Prompts are also normally what you would say to another engineer.

* There is a lot of duplication between A & B. Refactor this.

* Look at ticket X and give me a root cause

* Add support for three new types of credentials - Basic Auth, Bearer Token and OAuth Client Creds

Claude.md has stuff like "Here's how you run the frontend. here's how u run backend. This module support frontend. That module is batch jobs. Always start commit messages with ticket number. Always run compile at the top level. When you make code changes, always add tests" etc etc

Post reply on HN