61–70 of 98 posts
I have a MA system setup for personal use. You give it a problem, you then refine that problem where a fast, cheaper model asks you questions which you answer to get a better input prompt. You then choose a MA strategy for example take problem break up to sections then final judge concludes or you do multi turn where agents debate then judge summarises debate. The best approach is what I call 'all angles' where all t…
Earlier quoted context omitted.
And AWS heavily pushes a complex lambda solution stringing together as many chargeable AWS services as possible for a simple requirement Their interests are often not your interests. In this case they want you to unnecessary money on useless work (let's stop the euphemism of "tokens" btw)
This kind of cute conspiracy theories don’t actually hold true in real life. The companies want to make useful products.
/s
One month I could use Github Copilot fully with no disruptions. The next month, after pricing changes, I’ve run out of tokens in two days. Such drastic changes tell me that pricing of tokens is arbitrary, and AI business is running out of money fast.
Also I mean prices in generally for all things are based on underlying factors, that doesn't make them arbitary (i.e. github executives using a random number generator for token pricing would be arbitary)
First thought was "only 30 tasks" however the findings map to what I've seen personally: code review consumes majority of tokens
Code review could also be run as an unattended/batched task though, possibly with at least some use of on-prem inference (which excels at this). That would be a major saving compared to the usual cloud inference scenario.
One thing I've noticed using agents for coding is that they really like to write thousands of unit tests but not dynamically test.
you can just tell them to do more dynamic testing. I think dynamic testing is partly frowned upon because it slows things down & can take down software where you wouldn't expect
Earlier quoted context omitted.
So what harness are you using? And what LLM’s
Homebrew harness and all frontier ones plus deepseek. All via Openrouter at the moment. Works well enough but can get expensive so use for real high value challenges. Interestingly the refine feature has been most useful to me and people I have shown, essentially people are lazy when expressing the initial problem (me included!), refine asks relevant questions to initial problem then refines the initial statement, us…
I don't think a specific harness is even necessary to get a boost from 'Refine'. Even a simple custom agent is portable enough... it's easy enough to take the existing 'Plan' agent definition present in VS Code and tweak it to be 'Refine' instead.
I have a MA system setup for personal use. You give it a problem, you then refine that problem where a fast, cheaper model asks you questions which you answer to get a better input prompt. You then choose a MA strategy for example take problem break up to sections then final judge concludes or you do multi turn where agents debate then judge summarises debate. The best approach is what I call 'all angles' where all t…
Definitely interested, would love to see a video :)