Meh
I came here to say exactly this.
No git repo - no sharing - just another sign up and be AMAZED page.
Meh
Show HN: OzBrain, a shared brain for knowledge between agents and your team
61–70 of 73 posts
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#62I recently built our knowledge management system based (loosely) on the Karpathy LLM-Wiki model.
I'm not sure I understand why it is so important to be multi-agent, or for each agent to define who they are and what changes they made.
Our process uses git for tracking, so much of these details are captured natively in the merge.
Our process is
Drop a change, or something to document in an "inbox". Agent makes a branch, processes the inbox materials, updates the documents that need to be updated, updates timelines, makes a note of others on the team that may need to be notified of changes.
Creator reviews the PR and ensures the changes match how they want the document to be updated. Anyone that needs to be updated is now a reviewer on the PR.
I've just started rolling this out to the rest of the team, thankfully we are an engineering heavy organization so everyone is comfortable in Git.
The system keeps a running audit log, input is never deleted, it's stored, documents are updated, people are notified, etc. etc.
I'm just not sure what the service is offering, and how being multi-agent is solving a problem, but maybe I just haven't hit that problem yet.
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#63I might have a slightly different take. I think tools like Liner, or project management kanban boards like Trello, could actually be the best way for humans and AI to collaborate in the future. My own team of AI employees has already started working and collaborating in this way.
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#64[flagged]
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#65I recently built our knowledge management system based (loosely) on the Karpathy LLM-Wiki model. I'm not sure I understand why it is so important to be multi-agent, or for each agent to define who they are and what changes they made. Our process uses git for tracking, so much of these details are captured natively in the merge. Our process is Drop a change, or something to document in an "inbox". Agent makes a branch…
It sounds like you've got a great system for your technical folks. The question is if that works super well for sales, design, support, etc. I'm trying to build something that works equally well for the technical & non-technical folks. All connected to the same system. All in auto-pilot so nobody has to manage it.
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#66Great entry point! I've also run into this issue and am trying to solve it. But when I try this in different ways, a common issue is that the LLM often drifts. Sometimes it just drops important issues or records more trivial info than I need. Wondering how you solve this issue in OzBrain.
I'm not trying to solve the ingestion problem. It is there for sure, agents are pretty terrible at always following directions, so hopefully it's easy enough to just say "Put that in the brain" when in doubt. What is good is to save the source file ie transcription, raw comment/text, so that it is always there for future reference so the reasoned summary can be audited.
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#67Meh
I came here to say exactly this. No git repo - no sharing - just another sign up and be AMAZED page. Meh
Thank you for your insight. FYI You can share your brain with collaborators/partners. Did you read my comment on the post? You just clicked a link, looked at a homepage and got a meh-boner to come back and comment?
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#68[flagged]
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#69Do you have a solution for degradation in accuracy when compiling larger amounts of llm-produced text? I am also building LLM knowledge/memory systems and I've been surprised how bad LLMs are, even SOTA models, at summarizing non-trivial input batches of text. They get things wrong, distort the underlying meaning or data, etc.
Divide and conquer essentially, is what I've found so far to work best. Split things into smaller and smaller chunks to independently be verified, double-check everything, then coalesce upwards with verified summarizations. Have benchmarks for every single task and sub-task that will happen everywhere a LLM is involved, so you can measure improvements. Takes a ton more effort and tokens in the system itself obviously…
> Have benchmarks for every single task and sub-task that will happen everywhere a LLM is involved, so you can measure improvements
exactly. are there standard approaches or tools specifically for instrumenting LLM utilization, for applications rather than model development?
i use claude code a bit. i relentlessly push it to keep its work ephemera in source controlled and discoverable locations, following a lot of ozbrain's motivation. i also have claude generate task logs for all spawns. the theory with the task logs is to try and build some standard measures so i can see whether a thing i do (writing skills, experimenting with third-party MCP tooling, etc) actually helps in a way that is more robust than gut. i have yet to turn that effort into something useful. i'm just starting to get enough data to try to do some diving.
the problem is that this feels like the classic problem of "how do we measure productivity"? i was thinking very basic efficiency like "tokens in per tokens out" (maybe with a domain-specific value weighting function on the output tokens, e.g. code tokens > memory tokens > chat tokens) could be an possibility.
the point being, seems like you'd want to become pretty sophisticated about measurement to sell a product like ozbrain, as a basis for continuous improvement across major versions. or convincing people to use it, quantitatively rather than qualitatively.
Re: Show HN: OzBrain, a shared brain for knowledge between agents and your team
#70[flagged]