Building agents without harness engineering
rajitkhanna.com
Building agents without harness engineering
1–10 of 18 posts
Re: Building agents without harness engineering
#2Interesting idea! Question:
> It is highly unlikely that an AI agent startup becomes wealthy by creating the best harness for a particular use case.
If it's not the harness, what do you think is the thing that will differentiate AI agent startups? Is it mainly data, or something else?
Re: Building agents without harness engineering
#3I thought the entire industry is moving toward harness engineering? I read this twice and didn't fully understand what it was telling me.
Re: Building agents without harness engineering
#4I thought the entire industry is moving toward harness engineering? I read this twice and didn't fully understand what it was telling me.
Thanks for the feedback. The main idea is that today to built a best-in-class agent, developers build the agent loop, session management, tools, memory, skills, automations (cron + trigger-based), sandboxed deployment, and self-learning.
By providing Hermes with a system prompt, custom tools, and skills, developers get the agent loop, session management, automations, sandboxed deployment, and self-learning for free.
Re: Building agents without harness engineering
#5Interesting idea! Question: > It is highly unlikely that an AI agent startup becomes wealthy by creating the best harness for a particular use case. If it's not the harness, what do you think is the thing that will differentiate AI agent startups? Is it mainly data, or something else?
The most valuable pieces of information an AI agent startup can gather is access to their customer's proprietary data and knowledge of their customers preferences (memory + self-learning).
Even as the cost of writing code goes to zero, those two pieces of information are non-commodities.
Re: Building agents without harness engineering
#6If you re-use the Hermes agent, what are the cost and security implications? One Docker container per-customer sounds like it would be really expensive. Are they started on-demand, or run 24/7? What keeps users from using the agents for general purpose tasks, protects against prompt-injection, etc?
Re: Building agents without harness engineering
#7I'm curious who the ideal customer of this should be. If we're a startup with our own harness, are we a good fit? What would qualify us or disqualify us from being a good user?
Re: Building agents without harness engineering
#8But isn’t that the same as using Claude agent sdk minus maybe the memory features? What I mean to say is that you could pick the latest one and switch when another better one rolls out?
We’re using Claude agent sdk right now to rollout an internal agent factory. We haven’t hit the memory issue yet but I do use Hermes as a personal agent and can see where it fits you.
Re: Building agents without harness engineering
#9If you re-use the Hermes agent, what are the cost and security implications? One Docker container per-customer sounds like it would be really expensive. Are they started on-demand, or run 24/7? What keeps users from using the agents for general purpose tasks, protects against prompt-injection, etc?
> what are the cost and security implications?
Cost is the token usage and container uptime.
> One Docker container per-customer sounds like it would be really expensive.
The advantage is per-user memory and self-learning. For context, Claude Managed Agents uses one sandbox per session: https://platform.claude.com/docs/en/managed-agents/environme....
> Are they started on-demand, or run 24/7?
24/7 (best for customer-facing chat products).
> What keeps users from using the agents for general purpose tasks, protects against prompt-injection, etc?
Users define their agent with a system prompt, tool definitions, and skills (which separate a media generation agent from a people search agent). We use Openrouter which has a prompt injection detection feature: https://openrouter.ai/docs/guides/features/guardrails/prompt....
Re: Building agents without harness engineering
#10I'm curious who the ideal customer of this should be. If we're a startup with our own harness, are we a good fit? What would qualify us or disqualify us from being a good user?
Developers with customer-facing chat products are the ideal customer.
If a startup has a specific flow they want the agent to take and their traffic is bursty, then I'd recommend using a framework like Mastra and deploying onto a sandbox.
For long-running always on agents where it's important to learn the users preferences overtime, our approach is the highest ROI.