[dead]
Agents that run while I sleep
341–350 of 532 posts
Re: Agents that run while I sleep
#342This _all_ (waves hands around) sounds like alot of work and expense for something that is meant to make programming easier and cheaper. Writing _all_ (waves hands around various llm wrapper git repos) these frameworks and harnesses, built on top of ever changing models sure doesn't feel sensible. I don't know what the best way of using these things is, but from my personal experience, the defaults get me a looong wa…
> sounds like alot of work and expense for something that is meant to make programming easier and cheaper.
It's not more work; it's a convergence of roles. BA/PO/QA/SWE are merging.
AI has automated aspects of those roles that have made the traditional separation of concerns less desirable. A new hybrid role is emerging. The person writing these acceptance criteria can be the one guiding the AI to develop them.
So now we have dev-BAs or BA-devs or however you'd like to frame it. They're closer to the business than a dev might have been or closer to development than a BA might have been. The point is, smaller teams are able to play wider now.
Re: Agents that run while I sleep
#343This _all_ (waves hands around) sounds like alot of work and expense for something that is meant to make programming easier and cheaper. Writing _all_ (waves hands around various llm wrapper git repos) these frameworks and harnesses, built on top of ever changing models sure doesn't feel sensible. I don't know what the best way of using these things is, but from my personal experience, the defaults get me a looong wa…
I can't believe we're back to advocating for TDD. It was a failed paradigm that last few times we tried it. This time isn't any different because the fundamental flaw has always been the same: tests aren't proofs, they don't have complete coverage. Before anyone gets too confused, I love tests. They're great. They help a lot. But to believe they prove correctness is absolutely laughable. Even the most general tests a…
> But to believe they prove correctness is absolutely laughable.
Sounds like a lack of tests for the correct things.
Re: Agents that run while I sleep
#344Earlier quoted context omitted.
It feels like everyone's gone mad. Here I am mostly writing code by hand, with some AI assistant help. I have a Claude subscription but only use it occasionally because it can take more time to review and fix the generated code as it would to hand-write it. Claude only saves me time on a minority of tasks where it's faster to prompt than hand-write. And then I read about people spending hundreds or thousands of dolla…
Why read code when you are getting results fast ? See https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16d... I am not kidding. People don't seem to understand what's actually happening in our industry. See https://www.linkedin.com/posts/johubbard_github-eleutherailm...
These are like, jokes right?
Re: Agents that run while I sleep
#345Earlier quoted context omitted.
Why read code when you are getting results fast ? See https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16d... I am not kidding. People don't seem to understand what's actually happening in our industry. See https://www.linkedin.com/posts/johubbard_github-eleutherailm...
Why is everyone obsessed with Mac Minis. They're awesome but for the work that these people are attempting to do? Just seems... nonsensical. Renting a server is cheaper and still just as "local" as any of this (they want "self hosted", I don't think anyone cares about local. Like are people air gapping networks? lol) And a senior director of Nvidia? He had several Mac Minis? I really gotta imagine a Spark is better..…
It seems like the monkey-ladders story. Someone probably just had one sitting around and it worked or needed to do something Apple-specific and that message got lost along the way
Re: Agents that run while I sleep
#346This _all_ (waves hands around) sounds like alot of work and expense for something that is meant to make programming easier and cheaper. Writing _all_ (waves hands around various llm wrapper git repos) these frameworks and harnesses, built on top of ever changing models sure doesn't feel sensible. I don't know what the best way of using these things is, but from my personal experience, the defaults get me a looong wa…
> sounds like alot of work and expense for something that is meant to make programming easier and cheaper. It's not more work; it's a convergence of roles. BA/PO/QA/SWE are merging. AI has automated aspects of those roles that have made the traditional separation of concerns less desirable. A new hybrid role is emerging. The person writing these acceptance criteria can be the one guiding the AI to develop them. So no…
> It's not more work
It literally is. You're spending weeks of effort babysitting harnesses and evaluating models while shipping nothing at all.
Re: Agents that run while I sleep
#347Earlier quoted context omitted.
Sounds a lot like paying for online ads, they don't work because you're not paying enough, when in reality bots, scrapers and now agents are just running up all the clicks. You pay more to try and get above that noise and hope you'll reach an actual human. The new "fast mode" that burns tokens at 6 times the rate is just scary because that's what everyone still soon say we all need to be using to get results.
It feels like everyone's gone mad. Here I am mostly writing code by hand, with some AI assistant help. I have a Claude subscription but only use it occasionally because it can take more time to review and fix the generated code as it would to hand-write it. Claude only saves me time on a minority of tasks where it's faster to prompt than hand-write. And then I read about people spending hundreds or thousands of dolla…
I've been thinking about this recently and it seems like the most enthusiastic boosters always suggest difference in results is a skill issue, but I feel like there are 4 factors which multiply out to influence how much value someone gets:
- The quality of model output for _your particular domain / tech stack_. Models will always do better with languages and libraries they see a lot of than esoteric or proprietary
- The degree to which "works" = "good" in your scenario. For a one off script, "works" is all that matters, for a long lived core library, there are other considerations.
- The degree to which "works" can be easily (best yet, automatically) verified.
- Techniques, existing code cleanliness, documentation etc.
Boosters tend to lay all different experiences at the feet of this last, yet I'd argue the others are equally significant.
On the other hand, if you want to get the best results you can given the first 3 (which are generally out of one's control) then don't presume there's nothing you can do to improve the 4th.
Re: Agents that run while I sleep
#348Earlier quoted context omitted.
The trick is, with the setup I mentioned, you change the rewards. The concept is: Red Team (Test Writers), write tests without seeing implementation. They define what the code should do based on specs/requirements only. Rewarded by test failures. A new test that passes immediately is suspicious as it means either the implementation already covers it (diminishing returns) or the test is tautological. Red's ideal outco…
This is very interesting, but like sibling comments, I'm very curious as to how you run this in practice. Do you just tell Claude/Copilot to do what you describe? And do you have any prompts to share?
Sign up for your Claude Max (TM) subscription and have Claude set you up
Re: Agents that run while I sleep
#349This _all_ (waves hands around) sounds like alot of work and expense for something that is meant to make programming easier and cheaper. Writing _all_ (waves hands around various llm wrapper git repos) these frameworks and harnesses, built on top of ever changing models sure doesn't feel sensible. I don't know what the best way of using these things is, but from my personal experience, the defaults get me a looong wa…
I would encourage my competitors to use AI agents on their codebase as much as possible. Make sure every new feature has it, lots of velocity! Run those suckers day and night. Don't review it, just make sure the feature is there! Then when the music stops, the AI companies hit the economic realities, go insolvent, and they are left with no one who understands a sprawling tangled web of code that is 80% AI generated,…
> Don't review it, just make sure the feature is there!
Bad idea. Use another agent to do automatic review. (And a third agent writing tests.)
Don't forget the architecting and orchestrating agent too!
Re: Agents that run while I sleep
#350Earlier quoted context omitted.
The trick is, with the setup I mentioned, you change the rewards. The concept is: Red Team (Test Writers), write tests without seeing implementation. They define what the code should do based on specs/requirements only. Rewarded by test failures. A new test that passes immediately is suspicious as it means either the implementation already covers it (diminishing returns) or the test is tautological. Red's ideal outco…
This seems like a tremendous amount of planning, babysitting, verification, and token cost just to avoid writing code and tests yourself.
It just seems totally crazy to me, I don't understand how wrestling with this slot machine is even mentally easier