Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents that run while I sleep

claudecodecamp.com

Post reply on HN
Print view View original

151–160 of 532 posts

Re: Agents that run while I sleep

#151

Post by lagrange77 »

hexaga wrote: post #92
lagrange77 wrote: post #60

Earlier quoted context omitted.

> Reward hacking is very real and hard to guard against. Is it really about rewards? Im genuinely curious. Because its not a RL model.

There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model. And with that comes reward hacking - which isn't really about looking for more reward but rather that the model has learned patterns of behavior that got reward in the train env. That is, any kind of vulnerability in the train env manifests as something you'd recognize as reward hacking in the real world:…

> There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model.

Hm, as i understand it, parts of the training of e.g. ChatGPT could be called RL models. But the subject to be trained/fine tuned is still a seq2seq next token predictor transformer neural net.

Re: Agents that run while I sleep

#152

Post by skyberrys »

To me the last paragraph was the highest value in the article. Write out your test in plain language first, and then write the prompt for the autonomous agent using your language and the test prompt not the auto-code.

Re: Agents that run while I sleep

#153

Post by hinkley »

gedy wrote: post #140
wesselbindt wrote: post #133

Does anyone know what this guy is having his agents build? Bc I looked a bit and all I see him ship is linkedin posts about Claude.

Yeah maybe I'm just old but in 25 years in industry - not one company has needed this much code that fast. They may insist they do but then it sits while they figure out how to sell, or the inevitable "oh wait, we didn't think about that..."

Hurry up and wait.

Re: Agents that run while I sleep

#154

Post by 8note »

bluGill wrote: post #97
gchamonlive wrote: post #78

Earlier quoted context omitted.

It can if your refactor needs to deal with interface changes, like moving methods around, changing argument order etc... all these need to propagate to the tests

Your tests are an assertion that 'no matter what this will never change'. If your interface can change then you are testing implementation details instead of the behavior users care about. the above is really hard. A lot of tdd 'experts' don't understand is and teach fragile tests that are not worth having.

https://www.hyrumslaw.com/

your implementation is your interface. its a bit naive or hating-your-users to assume your tests are what your users care about. theyre dealing with everything, regardless of what youve tested or not.

Re: Agents that run while I sleep

#155

Post by hinkley »

overfeed wrote: post #107

> At some point you're not reviewing diffs at all, just watching deploys and hoping something doesn't break. To everyone who plan on automating themselves out of a job by taking the human element out- this is the endgame that management wants: replacing your (expensive and non-tax-optimized) labor with scalable Opex.

It's also delusional.

Re: Agents that run while I sleep

#157

Post by nojito »

canadiantim wrote: post #135
w4yai wrote: post #117

Earlier quoted context omitted.

You guys are describing wonderful things, but I've yet to see any implementation. I tried coding my own agents, yet the results were disappointing. What kind of setup do you use ? Can you share ? How much does it cost ?

Check out Mike Pocock’s work, he’s done excellent work writing about red green refactor and has a GitHub repo for his skills. Read and take what you need from his tdd skill and incorporate it into your own tdd skill tailored for your project.

This is just ai slop. If you follow what the actual designers of Claude/GPT tell you it flys in the face of building out over engineered harnesses for agents.

Re: Agents that run while I sleep

#158

Post by the_af »

xandrius wrote: post #112
paganel wrote: post #98

Earlier quoted context omitted.

> off work trees and running all the agents that I could afford, I still think that we, programmers, having to pay money in order to write code is a travesti. And I'm not talking about paying the license for the odd text editor or even for an operating system, I'm talking about day-to-day operations. I'm surprised that there isn't a bigger push-back against this idea.

It's silly, who wouldn't answer yes to the question "would you like to finish your task faster?". The real trick is to produce more but by putting less effort than before.

If you finish faster, you'll be given another task. You're not freeing yourself sooner or spending less effort, you're working the same number of hours for the same pay. Your reward is not joining the ranks of those laid off.

Re: Agents that run while I sleep

#159

Post by 8note »

afro88 wrote: post #89
aray07 wrote: post #35

Earlier quoted context omitted.

yeah honestly thats what i am struggling with too and I dont have a a good solution. However, I do think we are going to see more of this - so it will be interesting to see how we are going to handle this. i think we will need some kind of automated verification so humans are only reviewing the “intent” of the change. started building a claude skill for this ( https://github.com/opslane/verify )

It's a nice idea, but how do you know the agent is aligned with what it thinks the intent is?

or moreso, what happens at compact boundaries where the agent completely forgets the intent

Re: Agents that run while I sleep

#160

Post by joegaebel »

bhouston wrote: post #32

I call this "Test Theatre" and it is real. I wrote about it last year: https://benhouston3d.com/blog/the-rise-of-test-theater You have to actively work against it.

I've found the best way to achieve that is to force the agent to do TDD. Better to get it to do Outside-in TDD. Even better to get it to run Outside-in TDD, then use mutation testing to ensure it has fully covered the logic.

I've written about this and have a POC here for those interested: https://www.joegaebel.com/articles/principled-agentic-softwa...

Post reply on HN