The Harness Is the Thing • Hacker News Board

Re: The Harness Is the Thing

#91

Post by throw10920 » Thu, Aug 27, 2026, 1:57 PM UTC

dmantis wrote: post #85

dingaling911 wrote: post #81

Earlier quoted context omitted.

This problem is easily solved. "Check that the blog looks the same in XYZ browsers". I have LLMs align stuff for me all the time because I'm too lazy. It takes the screenshot, changes the code, code auto reloads and boom, done.

Sure. However I have a strange feeling when I read all AI praises and how productive everyone is while seeing more and more broken things everywhere. Sometimes from the same very productive people. Otherwise we might have said that maybe broken stuff goes from AI-deniers. If everyone is doing the job of hundreds, extremely productive and everything is so easily fixed, why everything feels so slow and broken, even so…

> If everyone is doing the job of hundreds, extremely productive and everything is so easily fixed, why everything feels so slow and broken, even so basic things?

This doesn't make sense. There were slow and broken things before AI. This was true even when the things were made by hundreds or thousands of engineers.

If an organization doesn't care about making their app or site fast or correct, then unless they have a truly ludicrous amount of free manpower (more than is available now with AI, because review and architecture are still bottlenecks), it probably won't happen.

AI doesn't change this. AI doesn't change the priorities of an organization, it just changes how quickly and cheaply they can build. And every org has a point in their priority queue where things are no longer worth it to spend resources on.

If Venmo didn't care about making their login page work well with Firefox (which, in my experience, they don't) before AI, why would you expect it to be better with AI?

Re: The Harness Is the Thing

#92

Post by criley2 » Thu, Aug 27, 2026, 2:05 PM UTC

bluegatty wrote: post #86

criley2 wrote: post #58

Earlier quoted context omitted.

Sonnet 5 is the worst model of 2026. Literally just turn effort slider down on Opus, it's smarter, faster and cheaper than whatever Sonnet is. Beyond that, I find this whole plan and build thing to be a pointless waste of tokens. If your planner made a detailed enough plan, then the cost of executing that plan is a just one turn more of cached tokens, and minimal time. Meanwhile: switching agents, reloading context a…

Completely disagree (except the Sonnet bit, yes, it's degrading). "then the cost of executing that plan is a just one turn more of cached tokens, and minimal time." This is just not true at all. There's a huge gap between 'figured out the hard stuff' and 'rock solid'. Dependencies, integration, corner cases, docs, testing, unforeseen issues, a lot of back and forth auditing making sure things are really tight. Audits…

I'm sorry, but just because you achieve results you consider acceptable with this method doesn't mean everyone does.

I don't work where we can ship slop. I don't work where PRs can be merged based on what the agents say. I work where a human has to read and approve and own every single line of code. I work where the stakes are actually high, so the cost of not using the best tools in terms of human time are big. A single turn around in a PR costs more in human time than the difference between deepseek and fable in API costs.

So, when you admit "There's a huge gap between 'figured out the hard stuff' and 'rock solid'." but then claim that the cheapest/dumbest agent in your arsenal is your go-to for "rock solid", I have to question the quality of your results.

Personally, "using plan mode" is a very 2025 way of using these tools, and I wouldn't be surprised to see "plan mode" be removed from codex/claude code/et al.

Realistically, I'm using the best models to think about a domain and problem (Fable High+), and I'm using a cheap daily driver with an advisor pattern (Opus High + Fable) to iterate through POCs, and I'm using human review to guide design. None of that is "plan mode", it's actual engineering. Then we decompose the solution, we stack it, and we use only really strong agents to build, review and refine.

This obsession with cheap agents leads to low quality outcomes. "Rock solid" deserves the best tools, and the "plan" will never be good enough. I'm going to be sending fable xhigh and sol 56 xhigh et al at it in adversarial review, why the heck am I cheaping out on the actual implementation?

And finally: my time costs way more than any of this. Cheaper models are slower overall and when combined with re-work time, are dramatically slower. I'm costing my company hundreds in my time to save a few bucks on the API bills. Nonsense!

Re: The Harness Is the Thing

#93

Post by adamtulinius » Thu, Aug 27, 2026, 2:09 PM UTC

tokai wrote: post #84

andunie wrote: post #65

I don't understand why no one has tried to make a harness without full shell access yet. It would be so much safer.

That exist? "permission": { "bash": "deny" } Or something equivalent in any agentic editor of your choice.

What does that achieve if the agent can still run processes on the host system?

Re: The Harness Is the Thing

#94

Post by trollbridge » Thu, Aug 27, 2026, 2:09 PM UTC

andunie wrote: post #66

trollbridge wrote: post #60

Earlier quoted context omitted.

No, but the 3 of us can now accomplish a lot more than we used to. We spend around $450 a month on hosted inference plus capital / utility costs of around $100 a month in local inference. And our 2 non-technical staff are now busy designing apps, so when they hand off something to be productised, it’s far more complete than the old days of a few Figma drawings

Are you sure you are? Try laying some people off just to be sure.

I'm very un-motivated to do so since then I'd have more work to do, and I'd be making less money.