Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents that run while I sleep

claudecodecamp.com

Post reply on HN
Print view View original

261–270 of 532 posts

Re: Agents that run while I sleep

#261

Post by datsci_est_2015 »

JumpCrisscross wrote: post #245
timr wrote: post #232

Earlier quoted context omitted.

> But it works well enough for most use cases. So does the code produced by any bad engineer. So either we’re finally admitting that all of that leetcode screening and engineer quality gating was a farce, or it wasn’t, and you’re wrong. I think the answer is in the middle, but the pendulum has swung too far in the “doesn’t matter” direction.

> we’re finally admitting that all of that leetcode screening and engineer quality gating was a farce, or it wasn’t, and you’re wrong We’re admitting a bit of both. Offshoring just became more instantaneous, secure and efficient. There will still be folks who overplay their hand. Macroeconomically speaking, I don’t see why we need more software engineers in the future than we have today, and that’s probably a conserv…

> Macroeconomically speaking, I don’t see why we need more software engineers in the future than we have today, and that’s probably a conservative estimate.

Why? Is the argument that there’s a finite amount of software that the world needs, and therefore we will more quickly reach that finite amount?

Seems more likely to me that if LLMs are a force multiplier for software then more software engineers will exist. Or, instead of “software engineers”, call them “people who create software” (even with the assistance of LLMs).

Or maybe the argument is that you need to be a super genius 100x engineer in order to manipulate 17 collaborative and competitive agents in order to reach your maximum potential, and then you’ll take everyone’s jobs?

Idk just seems like wild speculation that isn’t even worth me arguing against. Too late now that I’ve already written it out I guess.

Re: Agents that run while I sleep

#263

Post by godelski »

hi_hi wrote: post #206

This _all_ (waves hands around) sounds like alot of work and expense for something that is meant to make programming easier and cheaper. Writing _all_ (waves hands around various llm wrapper git repos) these frameworks and harnesses, built on top of ever changing models sure doesn't feel sensible. I don't know what the best way of using these things is, but from my personal experience, the defaults get me a looong wa…

I can't believe we're back to advocating for TDD. It was a failed paradigm that last few times we tried it. This time isn't any different because the fundamental flaw has always been the same: tests aren't proofs, they don't have complete coverage.

Before anyone gets too confused, I love tests. They're great. They help a lot. But to believe they prove correctness is absolutely laughable. Even the most general tests are very narrow. I'm sure they help LLMs just as they help us, but they're not some cure all. You have to think long and hard about problems and shouldn't let tests drive your development. They're guardrails for checking bonds and reduce footguns.

Oh, who could have guessed, Dijkstra wrote about program completeness. (No, this isn't the foolishness of natural language programming, but it is about formalism ;)

https://www.cs.utexas.edu/~EWD/transcriptions/EWD02xx/EWD288...

Re: Agents that run while I sleep

#265

Post by devinplatt »

egeozcan wrote: post #65
magicalist wrote: post #47

Earlier quoted context omitted.

> But even better, tell it to create subagents to form red team, green team and refactor team while the main instance coordinates them, respecting the clean-room rules. It really works. It helps, but it definitely doesn't always work, particularly as refactors go on and tests have to change. Useless tests start grow in count and important new things aren't tested or aren't tested well. I've had both Opus 4.6 and Code…

The trick is, with the setup I mentioned, you change the rewards. The concept is: Red Team (Test Writers), write tests without seeing implementation. They define what the code should do based on specs/requirements only. Rewarded by test failures. A new test that passes immediately is suspicious as it means either the implementation already covers it (diminishing returns) or the test is tautological. Red's ideal outco…

I'm curious how this works if the green team writes an implementation that makes a network call like an RPC.

Red team might not anticipate this if the spec does detail every expected RPC (which seems unreasonable: this could vary based on implementation). But a unit test would need mocks.

Is green team allowed to suggest mocks to add to the test? (Even if they can't read the tests themselves?) This also seems gamaeable though (e.g. mock the entire implementation). Unless another agent makes a judgement call on the reasonability of the mock (though that starts to feel like code review more generally).

Maybe record/replay tests could work? But there are drawbacks in the added complexity.

Re: Agents that run while I sleep

#267

Post by lelanthran »

jack_pp wrote: post #238
dheera wrote: post #235

Earlier quoted context omitted.

Not have to "build" anything. You edit code and it is already deployed on your dev instance. Deploying to production is just scp -rv * production:/var/www/ Beautifully simple. No npm build crap.

You trade having to compile for actually having code that can scale

It scales just fine the same way everything else scales: put a load balancer in front of multiple instances of your app.

Re: Agents that run while I sleep

#268

Post by imiric »

bluGill wrote: post #97
gchamonlive wrote: post #78

Earlier quoted context omitted.

It can if your refactor needs to deal with interface changes, like moving methods around, changing argument order etc... all these need to propagate to the tests

Your tests are an assertion that 'no matter what this will never change'. If your interface can change then you are testing implementation details instead of the behavior users care about. the above is really hard. A lot of tdd 'experts' don't understand is and teach fragile tests that are not worth having.

> Your tests are an assertion that 'no matter what this will never change'.

That's a strange definition. A lot of software should change in order to adapt to emerging requirements. Refactorings are often needed to make those changes easier, or to improve the codebase in ways that are transparent to users. This doesn't mean that the interfaces remain static.

> If your interface can change then you are testing implementation details instead of the behavior users care about.

Your APIs also have users. If you're only testing end-user interfaces, you're disregarding the users of your libraries and modules, e.g. your teammates and yourself.

Implementation details are contextual. To end-users, everything behind the external UI is an implementation detail. To other programmers, the implementation of a library, module, or even a single function can be a detail. That doesn't mean that its functionality shouldn't be tested. And, yes, sometimes that entails updating tests, but tests are code like any other, and also require maintenance and care.

Re: Agents that run while I sleep

#269

Post by godelski »

aprdm wrote: post #249
zarzavat wrote: post #210

Earlier quoted context omitted.

It feels like everyone's gone mad. Here I am mostly writing code by hand, with some AI assistant help. I have a Claude subscription but only use it occasionally because it can take more time to review and fix the generated code as it would to hand-write it. Claude only saves me time on a minority of tasks where it's faster to prompt than hand-write. And then I read about people spending hundreds or thousands of dolla…

Why read code when you are getting results fast ? See https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16d... I am not kidding. People don't seem to understand what's actually happening in our industry. See https://www.linkedin.com/posts/johubbard_github-eleutherailm...

Why is everyone obsessed with Mac Minis. They're awesome but for the work that these people are attempting to do? Just seems... nonsensical. Renting a server is cheaper and still just as "local" as any of this (they want "self hosted", I don't think anyone cares about local. Like are people air gapping networks? lol)

And a senior director of Nvidia? He had several Mac Minis? I really gotta imagine a Spark is better... at least it'll be a bit smarter of a cat (I'm pretty suspicious he used a LLM to help write that post)

No time to think, gotta go fast?

Re: Agents that run while I sleep

#270

Post by saguntum »

tudelo wrote: post #212
mjrbrennan wrote: post #144

Earlier quoted context omitted.

Yes I'm still not really understanding this "run agents overnight" thing. Most of the time if I use claude it's done in 5-20 minutes. I've never wanted to have work done for me overnight...tomorrow is already plenty of time for more work, it's not going anywhere, and my employer isn't paying me to produce overnight.

The only counter I have to this is that there are some workflows that have test environments, everything can't or shouldn't just run locally. Sometimes these test take time, and instead of babysitting the model to write code and run the build+deploy+test manually, you can send it off to work until the kinks are worked out. Add to that I have worked on many projects that take more than 20 minutes to fully build and ru…

Yeah, our basic integration test suite takes over 20 minutes to run in CI, likely higher locally but I never try to run the full test suite locally. That doesn't even encapsulate PDVs and other continuous testing that runs in the background.

The other day, I wrote a claude skill to pull logs for failing tests on a PR from CI as a CSV for feeding back into claude for troubleshooting. It helped with some debugging but was very fraught and needed human guidance to avoid going in strange directions. I could see this "fix the tests" workflow instrumented as overnight churn loops that are forbidden from modifying test files that run and have engineers review in the morning if more tests pass.

Maybe agentic TDD is the future. I have a bit of a nightmare vision of SWEs becoming more like QA in the future, but with much more automation. More engineering positions may become adversarial QA for LLM output. Figure out how to break LLM output before it goes to prod. Prove the vibe coded apps don't scale.

In the exercise I described above, I was just prompt churning between meetings (having claude record its work and feeding it to the next prompt, pulling test logs in between attempts), without much time to analyze, while another engineer on my team was analyzing and actually manually troubleshooting the vibe coded junk I was pushing up, but we fixed over 100 failing integration tests in a week for a major refactor using claude plus some human(s) in the loop. I do believe it got things done faster than we would have finished without AI. I do think the quality is slightly lower than would have been if we'd had 4 weeks without meetings to build the thing, but the tests do now pass.

Post reply on HN