Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Harness Is the Thing

scott-fryxell.github.io

Post reply on HN
Print view View original

51–60 of 102 posts

Re: The Harness Is the Thing

#51

Post by porridgeraisin »

vb-8448 wrote: post #48
nullbio wrote: post #28

LoRA adaptive learning using open-weight models and your own reasoning traces is the thing. The big labs have a mammoth job ahead of them if they want to compete with running your own model - they will basically have to give every single user their own persistent virtual machine. When it's all said and done, I think their only really moat will be as inference/hardware providers. Stripe buying OpenRouter was a very sm…

Can you explain what you mean with "LoRA adaptive learning using open-weight models and your own reasoning traces"?

You train an addend to the weights rather than the weights themselves. You limit the dimensions[1] available in this addend to a small fraction of the dimensions of the weights, making training cheaper.

When you have a few example chats you want a model to emulate - say you made it from your proprietary data, you can train any open model on that data in this cheap way. You don't lose any quality versus not using lora since the models overall knowledge won't shift that much due to your data anyways, so it's a waste to make high dimensional updates.

However, only in some cases is it worth it and equal in quality to just making a good retrieval system and exposing it to claude code or whatever. If a retrieval system over the same data is very difficult, or if the data simply must be proprietary, then you should go for it.

[1] technically "rank", but I'm simplifying

Re: The Harness Is the Thing

#52

Post by porridgeraisin »

slopinthebag wrote: post #12

Heh. Interesting article. > Single developer projects can build to the caliber and consistency of large development teams. This has always been true. Good developers, like truly good devs, could run rings around a team of mediocre devs. It’s a multiplier, a team of 10 1x devs will get dominated by a single 10x dev no matter how much AI they use. Nothing has changed here, if anything it benefits the good developers. >…

> Anything more complex is overentineering.

For general workflows, I agree. Doing some DAG system is just a waste of time if you don't have a grounded verifier for every node. If you are going to human verify, why make a dag of subagents and waste time? Keep the dag in your head, the way we all did before LLMs. The dag in your head is also much better.

For specific workflows however, even this is under engineering. If your specific task or family of tasks can truly be decomposed into multiple verifiable subtasks then you should spend the effort to build that system. It doesn't matter if it's technically worse than the other method because it is basically automatic and incredibly cheap. For complex systems this is a lot of upfront work, but if it is done, then the return is completely outsized.

I painted it black and white, but you can of course compose everything.

Re: The Harness Is the Thing

#53

Post by l33tman »

copperx wrote: post #18
jwrallie wrote: post #11

Earlier quoted context omitted.

It seems there are two distinct groups, one which is doing relatively well with Luna, Deepseek Flash and similar models while other seem to be satisfied only with the state of the art.

I'm ok with not-Fable until I need anything design-related: a nice HTML page, LaTeX typesetting, UI design. Sol is especially incapable of doing anything sensible. Is there any alternative model with design sensibilities?

Did you add some skill for this? I've been using a frontend design skill with Sol and it's reasonable (for my purposes). I don't remember where I got that skill.md though, it probably was from OAI's own blog about this a few months ago.

Re: The Harness Is the Thing

#54

Post by throw1234567891 »

douglee650 wrote: post #49

Author states, “Single developer projects can build to the caliber and consistency of large development teams.” When I, as a single person, can produce a project in one month that would have taken a team of four people three months to produce, why would I care about token cost? I’m now spending $500/month instead of $40,000 month to get the same thing 3x faster. $500 for a project instead of $120,000. (Assumes my cos…

Have you laid those people off? You still have people on the payroll. If each one of them uses a $500 subscription, you are now spending $42k/month, not $40k.

Re: The Harness Is the Thing

#55

Post by ozim »

throw1234567891 wrote: post #54
douglee650 wrote: post #49

Author states, “Single developer projects can build to the caliber and consistency of large development teams.” When I, as a single person, can produce a project in one month that would have taken a team of four people three months to produce, why would I care about token cost? I’m now spending $500/month instead of $40,000 month to get the same thing 3x faster. $500 for a project instead of $120,000. (Assumes my cos…

Have you laid those people off? You still have people on the payroll. If each one of them uses a $500 subscription, you are now spending $42k/month, not $40k.

Now think about that he is counting it like:

He never had to hire anyone in the first place.

He didn't have to make any contracts, deal with screening, background checks, recruitment calls.

All that money he didn't spend he can spend on maxing out token usage.

Re: The Harness Is the Thing

#56

Post by theahura »

If you are using any of the agents without config, do yourself a favor and invest in any config library that implements SPACE[0]

It will save you a ton of time and you won't be tied to fable costs

[0] Search, Plan, Assert, Code, Evaluate. Obras superpowers, Matt pococks skills, and the nori high autonomy skillset all have this built in

Re: The Harness Is the Thing

#57

Post by jacobfehn »

copperx wrote: post #18
jwrallie wrote: post #11

Earlier quoted context omitted.

It seems there are two distinct groups, one which is doing relatively well with Luna, Deepseek Flash and similar models while other seem to be satisfied only with the state of the art.

I'm ok with not-Fable until I need anything design-related: a nice HTML page, LaTeX typesetting, UI design. Sol is especially incapable of doing anything sensible. Is there any alternative model with design sensibilities?

fable is good at it but I find it uses the same slop in different variants every time. Even the words it uses for the different alternatives it proposes are the same (cartography, atlas, bench, signal, etc). At least you develop a good nose for slop ;) it becomes a struggle to steer it off the same path every time.

Re: The Harness Is the Thing

#58

Post by criley2 »

bluegatty wrote: post #42
AirMax98 wrote: post #3

Reading this really makes me wish that I had a slightly better workflow. I'm really soley dependent of Fable to the point that I don't use other models, and I've already sort of hit a point where I'm running into usage limits every week. I am really living on borrowed time — when Anthropic finally collapses their 50% usage increase at the end of August, I'll definitely be forced to switch my workflow. When that happe…

Just plan extensively and let Sonnet do the exec. And Opus5 aggressive audits. Once it has exactly your coding conventions and access to other code to copy bespoke patterns, a strong idea for what to do, then you can let it do the work. You do the wiring, it fills it in. Coding was never the work.

Sonnet 5 is the worst model of 2026. Literally just turn effort slider down on Opus, it's smarter, faster and cheaper than whatever Sonnet is.

Beyond that, I find this whole plan and build thing to be a pointless waste of tokens. If your planner made a detailed enough plan, then the cost of executing that plan is a just one turn more of cached tokens, and minimal time.

Meanwhile: switching agents, reloading context and building from the plan will easily balloon your token use and time. And any emergent problem that the dumb executor finds will instantly wreck the implementation because they're not competent at solving it. And if your plan is so perfect that there's no edge case then you're wasting tokens because your planner was one turn away from finishing the project via cached tokens.

Re: The Harness Is the Thing

#59

Post by s_dev »

douglee650 wrote: post #49

Author states, “Single developer projects can build to the caliber and consistency of large development teams.” When I, as a single person, can produce a project in one month that would have taken a team of four people three months to produce, why would I care about token cost? I’m now spending $500/month instead of $40,000 month to get the same thing 3x faster. $500 for a project instead of $120,000. (Assumes my cos…

"A robot that's a tenth of the speed is still cost effective when it's a hundredth of the price"

-- CPG Grey, Humans Need not Apply. Released over a decade ago.

https://www.youtube.com/watch?v=CMFj75kBQlU

Re: The Harness Is the Thing

#60

Post by trollbridge »

throw1234567891 wrote: post #54
douglee650 wrote: post #49

Author states, “Single developer projects can build to the caliber and consistency of large development teams.” When I, as a single person, can produce a project in one month that would have taken a team of four people three months to produce, why would I care about token cost? I’m now spending $500/month instead of $40,000 month to get the same thing 3x faster. $500 for a project instead of $120,000. (Assumes my cos…

Have you laid those people off? You still have people on the payroll. If each one of them uses a $500 subscription, you are now spending $42k/month, not $40k.

No, but the 3 of us can now accomplish a lot more than we used to. We spend around $450 a month on hosted inference plus capital / utility costs of around $100 a month in local inference.

And our 2 non-technical staff are now busy designing apps, so when they hand off something to be productised, it’s far more complete than the old days of a few Figma drawings

Post reply on HN