Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Harness Is the Thing

scott-fryxell.github.io

Post reply on HN
Print view View original

21–30 of 97 posts

Re: The Harness Is the Thing

#21

Post by packetlost »

jwrallie wrote: post #11
all2 wrote: post #4

Earlier quoted context omitted.

I've been using GPT-5.6 Luna for everything at this point. It is more than enough to do all the work I need to do. You have to lead it by the nose, but if you know where you're going it works really, really well.

It seems there are two distinct groups, one which is doing relatively well with Luna, Deepseek Flash and similar models while other seem to be satisfied only with the state of the art.

I'm happily in both depending on what it is. Even Sol / Fable cannot do some truly novel stuff and if you rely on it too heavily you get detached from the underlying systems to the point that it's both uncomfortable and detrimental.

Re: The Harness Is the Thing

#22

Post by coder-3 »

jwrallie wrote: post #11
all2 wrote: post #4

Earlier quoted context omitted.

I've been using GPT-5.6 Luna for everything at this point. It is more than enough to do all the work I need to do. You have to lead it by the nose, but if you know where you're going it works really, really well.

It seems there are two distinct groups, one which is doing relatively well with Luna, Deepseek Flash and similar models while other seem to be satisfied only with the state of the art.

Depends on how in-the-loop you want to be. I personally delegate all the writing of the code to agents, but I maintain a clear mental model of the architecture, which I come up with by iterating and prototyping with agents. I can use Luna for all of this, although I switch depending on the task. It's nice to be able to throw a 1,000 word rough spec at Fable and get a personal tool that works perfectly though. I expect that as the models get better, I'll continue to be in the loop, but operate at increasingly higher levels of abstraction.

Re: The Harness Is the Thing

#23

Post by esalman »

What I've learned in last week is that a harness is basically a while loop.

In each iteration you make an LLM call, perform some work (e.g. tool call), augment the prompt (append or compact etc.)- not necessarily in that other- and continue.

Until an end condition is satisfied. Then you break out.

Re: The Harness Is the Thing

#24

Post by dnnehgf »

i doubt it. someday harnesses get complex enough that you start asking models to interpret and modify them.

the pattern of layering of deterministic, probabilistic, deterministic, probabilistic. it's a strange pattern but it seems somehow natural.

Re: The Harness Is the Thing

#25

Post by edg5000 »

AirMax98 wrote: post #3

Reading this really makes me wish that I had a slightly better workflow. I'm really soley dependent of Fable to the point that I don't use other models, and I've already sort of hit a point where I'm running into usage limits every week. I am really living on borrowed time — when Anthropic finally collapses their 50% usage increase at the end of August, I'll definitely be forced to switch my workflow. When that happe…

Sol is basically at the same level, but with better usage. You're not dependent on Fable. You could switch the next minute and hardly experience quality differences.

Re: The Harness Is the Thing

#26

Post by visarga »

esalman wrote: post #23

What I've learned in last week is that a harness is basically a while loop. In each iteration you make an LLM call, perform some work (e.g. tool call), augment the prompt (append or compact etc.)- not necessarily in that other- and continue. Until an end condition is satisfied. Then you break out.

A pluggable loop with events and hooks.

Re: The Harness Is the Thing

#28

Post by nullbio »

LoRA adaptive learning using open-weight models and your own reasoning traces is the thing. The big labs have a mammoth job ahead of them if they want to compete with running your own model - they will basically have to give every single user their own persistent virtual machine. When it's all said and done, I think their only really moat will be as inference/hardware providers. Stripe buying OpenRouter was a very smart bet.

Re: The Harness Is the Thing

#29

Post by slopinthebag »

alansaber wrote: post #15
slopinthebag wrote: post #12

Heh. Interesting article. > Single developer projects can build to the caliber and consistency of large development teams. This has always been true. Good developers, like truly good devs, could run rings around a team of mediocre devs. It’s a multiplier, a team of 10 1x devs will get dominated by a single 10x dev no matter how much AI they use. Nothing has changed here, if anything it benefits the good developers. >…

Is the article slightly naive? Sure. It describes a relatively uncomplicated multi-agent system (which is good). But frontier model handing off a DAG rather than a raw markdown to a cheap model seems like a decent idea. Ultimately, this is about cost saving, not peak performance.

> But frontier model handing off a DAG rather than a raw markdown to a cheap model seems like a decent idea

It's not

Re: The Harness Is the Thing

#30

Post by mikestorrent »

noobcoder wrote: post #7
AirMax98 wrote: post #3

Reading this really makes me wish that I had a slightly better workflow. I'm really soley dependent of Fable to the point that I don't use other models, and I've already sort of hit a point where I'm running into usage limits every week. I am really living on borrowed time — when Anthropic finally collapses their 50% usage increase at the end of August, I'll definitely be forced to switch my workflow. When that happe…

Boy you should try model routing, I dont think using Fable for all the tasks makes sense to me

It reads like a flex. Online equivalent of a driver of a fancy car who has to make sure you know it.
Post reply on HN