I think we’re still figuring out the right abstraction for offering agents as a product. - LLMs are a great foundation but building your own harness is a huge undertaking, a deep rabbit hole. - There are harnesses available as open source libraries but that’s still coupled to an environment. Where does the state persist? Like maybe I’m a Cloudflare worker and don’t even have a file system. Agent as a service like thi…
OpenAI Agents API
161–170 of 201 posts
Re: OpenAI Agents API
#162I believe is not a good idea for the model providers provides a Agent Infrastructure or hosting service. This area should be open source and supported by the cloud providers. Because I don't really want to be locked to a model provider when I am building my agents. Actually this is happening, I found couples: - https://flueframework.com from Astra - https://eve.dev from Vercel - https://fastagent.sh looks more indepe…
Instead they'll point you to their respective vendor APIs for the specific use cases.
We're talking about trillions in value to be captured, they'll try everything.
Re: OpenAI Agents API
#163I think the line between regular LLM "endpoints" and agents/harnesses is going to become more and more blurry until it's a meaningless distinction. When you're using ChatGPT/Claude/Gemini etc. you're basically already interacting with some backend harness with tools etc., not a raw LLM. Just give it a computer and be done with it. I already find myself using Claude Code / Antigravity (via web) instead of Claude / Gem…
Because in most of those API, even many implementations of the Responses API, you lose a lot of control of where your data is going. e.g. an Agent or the Responses API may automatically invoke a tool call that leaks your data to an external service on the internet, without having an option to intervene.
If you want to have control over your data, you have to have control over your harness.
Re: OpenAI Agents API
#164Earlier quoted context omitted.
> building your own harness is a huge undertaking, a deep rabbit hole. I eventually gave up on this task. It's not possible to fight OpenAI or Anthropic's engineering teams. Their reasoning models have all kinds of undocumented back door access to the base models that you'd never be able to replicate from the outside. Even if you had full access you would not have the engineering man hours or experience to keep up. I…
I built my harness in pi within herdr, I cloned (zipped and downloaded) 0xRichardH/pi-herdr-subagents and went from there, and used pi to build itself, adding gate workflow state control, provider fallbacks (I use many token plans), subagent skill injection, etc. It is highly custom to my needs and wants, and I think every developer needs to do this. I only talk to my planner which plans, and it subs out to designer,…
I have been running my own setup and evolving it since November as I come up with more ideas and needs. The nicest bit is not being tied to one provider or another. Anthropic decides to stop supporting Claude agent sdk with subs? Or astra outstrips other models? My harness now works with Claude/cursor/codex and soon ACP. It didn’t start that way, it was originally all in on Claude, but now all the things I built on top of Claude now work with the others too.
Re: OpenAI Agents API
#165Re: OpenAI Agents API
#166Earlier quoted context omitted.
You are wrong. I honestly am having trouble understanding how you think these things work, and what you believe a harness is in the first place. There is the standard API that things like openRouter exposes, thats supported by every provider, and all it is, is manipulating a large blob of memory by adding your own words or asking the GPU to append new words. Everything else around it is window dressing. All providers…
Sorry, I didn’t mean to imply that what you’re doing is wrong, although re-reading my own comment sounds like so. The capabilities of a model unlocks certain harness behaviour, and in its terms might be beneficial to automate more of your x, y, z. I guess computer use would be an example I was thinking about. Certain models are not as good as some current models (Astra/Fable) in understanding through screenshots, or…
Not really.
Sure its nice for a model to nativly be trained on understanding browsers & DOM/js model, so it can effectively use a `browser` tool.
A model needs a certain 3d mental model before it can successfully use a `blender` tool. Opus 4.5 had this to some extend without specifically training for it - but sure this is one aspect i think models will rapidly improve on this year.
But at the same time these tools carry a description of how they're used specifically - so changing out the `blender` tool for some other 3d modeling software shouldn't affect its capabilities. Using/creating a coherent description of the desired art-style is most of the value of the work.
So its still not clear to me what you mean with "certain harness behavior" or "unlock new capabilities".
You get access to a model that can be better or worse at certain tasks, sure.
The big change i've noticed last year is they've gotten better at prompting themselves (subagents, delegate, w/e you want to call it).
But the explosion of how fast things went last year is mostly "existing software * better model". For that capability explosion to continue you'd need ever more "software" that improve with AI (or more existing software unblocking their interface for model use)
There will definitely be more software, but you need time to incorporate it in your work (like eg `git`) for it to meaningfully feels like AI is unlocking new things when using it.
For new stuff like being able to control a robot dog, we first all need to have a robot dog for it to feel like very fast development.
----
As for "Computer use" specifically that seems like a bad example from my perspective.
The only real "added capabilities" over the past year i've seen is better selfprompt + comprehending screenshots.
Every other use case i'd consider computer-use is just friction that needn't have been there in the first place. Its the software/os you're using that doesnt have a proper interface that's creating the friction. Not the lack of model/harness capabilities.
I have had a `browser` tool for a year that's just a debug-socket to ff/chrome with eval + pre/post screenshot, and ever since 4.4 ~ 4.5 every model has understood it just fine and can do basically anything.
Reasoning _what_ to do with it is the 'intelligence' we're trying to benchmark and everything since Opus 4.5 was basically 'good enough' for anything non-creative a human could do.
E.g. having a model get good at creating drafts of emails based on support tickets, is a function of how good the cli/mcp and instructions are.
Can you be more specific in what capability increase you see getting unlocked?
As a final point, the models are so good now that you can ask claude inside claude to write/setup plugins for pi to do everything claude can do.
The reason i dont use pi to ask to improve the claude harness, is because anthropic is trying to build a moat with their closed source crap. They'll succeed because enterprise is slow to adapt and doesn't understand its wasting money, not because their models unlock certain harness behavior. The competition who is not wasting money will have a step up.
Re: OpenAI Agents API
#167Earlier quoted context omitted.
Correct me if I'm wrong, but the harness will always be dependent on the underlying model, and useless without it. All custom harnesses are being built, could be obsolete in the next big-generation-jump of the models. I might be absolutely wrong, but "harnesses" / cc-derivatives became "good enough" only maybe a year ago max. Before that, people were pushing for gigantic folder structures with custom documents and "p…
A lot of what a custom harness does isn't necessarily interacting with a model, it's the framework around it. Part of harness engineering is figuring out what can be done deterministically to avoid calling an LLM. A generational jump doesn't obsolete those areas, it just means you get the right answer (maybe) faster and easier when you do call the model. There's a difference between their harness and your harness. Th…
Again, this is all subjective risk assessment. I very much support any engineer to learn how to make harnesses, and I, myself, have a long way to go. We’ll see where we are in a year or two, I guess.
Re: OpenAI Agents API
#168Re: OpenAI Agents API
#169Why on earth would we want such a lock-in at this stage when there is no clear winner. This is an area in which I would encourage everyone to build their own (using OSS) on top of existing cloud infrastructure.
Re: OpenAI Agents API
#170Earlier quoted context omitted.
Yet i find claude code worse than opencode.
I don't think anyone who's tried something else can like claude code. The only reason to use it is the subsidized plan and nothing else.
I could patch this together with Herdr and OMP and some custom server stuff but it works right out of the box!?